{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1任务描述 \n",
    "利用线性回归技术实现共享单车数量预测\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 数据说明\n",
    "\n",
    "Capital Bikesh are（美国Washington, D.C.的一个共享单车公司）提供的共享单车数据。数据包含每天的日期、天气等信息，需要预测每天的共享单车骑行量。\n",
    "\n",
    "1)文件说明\n",
    "\n",
    "day.csv: 按天计的单车共享次数（作业只需使用该文件）\n",
    "\n",
    "hour.csv: 按小时计的单车共享次数（无需理会）\n",
    "\n",
    "readme：数据说明文件\n",
    "\n",
    "2)字段说明\n",
    "\n",
    "Instant：记录号\n",
    "\n",
    "Dteday：日期\n",
    "\n",
    "Season：季节（1=春天、2=夏天、3=秋天、4=冬天）\n",
    "\n",
    "yr：年份，(0: 2011, 1:2012)\n",
    "\n",
    "mnth：月份( 1 to 12)\n",
    "\n",
    "hr：小时 (0 to 23)  （只在hour.csv有，作业忽略此字段）\n",
    "\n",
    "holiday：是否是节假日（0/1）\n",
    "\n",
    "weekday：星期中的哪天，取值为0～6\n",
    "\n",
    "workingday：是否工作日（0/1）\n",
    "\n",
    "1=工作日 （是否为工作日，1为工作日，0为非周末或节假日）\n",
    "\n",
    "weathersit：天气（1：晴天，多云 ",
    "2：雾天，阴天 ",
    "3：小雪，小雨 ",
    "4：大雨，大雪，大雾）\n",
    "\n",
    "temp：气温摄氏度\n",
    "\n",
    "atemp：体感温度\n",
    "\n",
    "hum：湿度\n",
    "\n",
    "windspeed：风速\n",
    "\n",
    "casual：非注册用户贡献的骑行量（作业无需理会该字段）\n",
    "\n",
    "registered：注册用户贡献的骑行量（作业无需理会该字段）\n",
    "\n",
    "cnt：给定日期（天, day.csv）时间（每小时,hour.csv）总租车人数，响应变量y\n",
    "\n",
    "\n",
    "casual、registered和cnt三个特征均为要预测的y（cnt =casual+registered ），作业里只需对cnt进行预测。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3特征工程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1导入相关工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd #SQL数据处理\n",
    "import numpy as np #矩阵\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2数据加载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>dteday</th>\n",
       "      <th>season</th>\n",
       "      <th>yr</th>\n",
       "      <th>mnth</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weekday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weathersit</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.344167</td>\n",
       "      <td>0.363625</td>\n",
       "      <td>0.805833</td>\n",
       "      <td>0.160446</td>\n",
       "      <td>331</td>\n",
       "      <td>654</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2011-01-02</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.363478</td>\n",
       "      <td>0.353739</td>\n",
       "      <td>0.696087</td>\n",
       "      <td>0.248539</td>\n",
       "      <td>131</td>\n",
       "      <td>670</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>2011-01-03</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.196364</td>\n",
       "      <td>0.189405</td>\n",
       "      <td>0.437273</td>\n",
       "      <td>0.248309</td>\n",
       "      <td>120</td>\n",
       "      <td>1229</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>2011-01-04</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>0.212122</td>\n",
       "      <td>0.590435</td>\n",
       "      <td>0.160296</td>\n",
       "      <td>108</td>\n",
       "      <td>1454</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>2011-01-05</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.226957</td>\n",
       "      <td>0.229270</td>\n",
       "      <td>0.436957</td>\n",
       "      <td>0.186900</td>\n",
       "      <td>82</td>\n",
       "      <td>1518</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant      dteday  season  yr  mnth  holiday  weekday  workingday  \\\n",
       "0        1  2011-01-01       1   0     1        0        6           0   \n",
       "1        2  2011-01-02       1   0     1        0        0           0   \n",
       "2        3  2011-01-03       1   0     1        0        1           1   \n",
       "3        4  2011-01-04       1   0     1        0        2           1   \n",
       "4        5  2011-01-05       1   0     1        0        3           1   \n",
       "\n",
       "   weathersit      temp     atemp       hum  windspeed  casual  registered  \\\n",
       "0           2  0.344167  0.363625  0.805833   0.160446     331         654   \n",
       "1           2  0.363478  0.353739  0.696087   0.248539     131         670   \n",
       "2           1  0.196364  0.189405  0.437273   0.248309     120        1229   \n",
       "3           1  0.200000  0.212122  0.590435   0.160296     108        1454   \n",
       "4           1  0.226957  0.229270  0.436957   0.186900      82        1518   \n",
       "\n",
       "    cnt  \n",
       "0   985  \n",
       "1   801  \n",
       "2  1349  \n",
       "3  1562  \n",
       "4  1600  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#读取数据\n",
    "data = pd.read_csv(\"day.csv\")\n",
    "\n",
    "#观察前5行\n",
    "data.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 731 entries, 0 to 730\n",
      "Data columns (total 16 columns):\n",
      "instant       731 non-null int64\n",
      "dteday        731 non-null object\n",
      "season        731 non-null int64\n",
      "yr            731 non-null int64\n",
      "mnth          731 non-null int64\n",
      "holiday       731 non-null int64\n",
      "weekday       731 non-null int64\n",
      "workingday    731 non-null int64\n",
      "weathersit    731 non-null int64\n",
      "temp          731 non-null float64\n",
      "atemp         731 non-null float64\n",
      "hum           731 non-null float64\n",
      "windspeed     731 non-null float64\n",
      "casual        731 non-null int64\n",
      "registered    731 non-null int64\n",
      "cnt           731 non-null int64\n",
      "dtypes: float64(4), int64(11), object(1)\n",
      "memory usage: 91.4+ KB\n"
     ]
    }
   ],
   "source": [
    "#查看数据信息\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 结论：数据完整无缺失"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3数据探索"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season</th>\n",
       "      <th>yr</th>\n",
       "      <th>mnth</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weekday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weathersit</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>366.000000</td>\n",
       "      <td>2.496580</td>\n",
       "      <td>0.500684</td>\n",
       "      <td>6.519836</td>\n",
       "      <td>0.028728</td>\n",
       "      <td>2.997264</td>\n",
       "      <td>0.683995</td>\n",
       "      <td>1.395349</td>\n",
       "      <td>0.495385</td>\n",
       "      <td>0.474354</td>\n",
       "      <td>0.627894</td>\n",
       "      <td>0.190486</td>\n",
       "      <td>848.176471</td>\n",
       "      <td>3656.172367</td>\n",
       "      <td>4504.348837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>211.165812</td>\n",
       "      <td>1.110807</td>\n",
       "      <td>0.500342</td>\n",
       "      <td>3.451913</td>\n",
       "      <td>0.167155</td>\n",
       "      <td>2.004787</td>\n",
       "      <td>0.465233</td>\n",
       "      <td>0.544894</td>\n",
       "      <td>0.183051</td>\n",
       "      <td>0.162961</td>\n",
       "      <td>0.142429</td>\n",
       "      <td>0.077498</td>\n",
       "      <td>686.622488</td>\n",
       "      <td>1560.256377</td>\n",
       "      <td>1937.211452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.059130</td>\n",
       "      <td>0.079070</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.022392</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>22.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>183.500000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.337083</td>\n",
       "      <td>0.337842</td>\n",
       "      <td>0.520000</td>\n",
       "      <td>0.134950</td>\n",
       "      <td>315.500000</td>\n",
       "      <td>2497.000000</td>\n",
       "      <td>3152.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>366.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.498333</td>\n",
       "      <td>0.486733</td>\n",
       "      <td>0.626667</td>\n",
       "      <td>0.180975</td>\n",
       "      <td>713.000000</td>\n",
       "      <td>3662.000000</td>\n",
       "      <td>4548.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>548.500000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.655417</td>\n",
       "      <td>0.608602</td>\n",
       "      <td>0.730209</td>\n",
       "      <td>0.233214</td>\n",
       "      <td>1096.000000</td>\n",
       "      <td>4776.500000</td>\n",
       "      <td>5956.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>731.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.861667</td>\n",
       "      <td>0.840896</td>\n",
       "      <td>0.972500</td>\n",
       "      <td>0.507463</td>\n",
       "      <td>3410.000000</td>\n",
       "      <td>6946.000000</td>\n",
       "      <td>8714.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          instant      season          yr        mnth     holiday     weekday  \\\n",
       "count  731.000000  731.000000  731.000000  731.000000  731.000000  731.000000   \n",
       "mean   366.000000    2.496580    0.500684    6.519836    0.028728    2.997264   \n",
       "std    211.165812    1.110807    0.500342    3.451913    0.167155    2.004787   \n",
       "min      1.000000    1.000000    0.000000    1.000000    0.000000    0.000000   \n",
       "25%    183.500000    2.000000    0.000000    4.000000    0.000000    1.000000   \n",
       "50%    366.000000    3.000000    1.000000    7.000000    0.000000    3.000000   \n",
       "75%    548.500000    3.000000    1.000000   10.000000    0.000000    5.000000   \n",
       "max    731.000000    4.000000    1.000000   12.000000    1.000000    6.000000   \n",
       "\n",
       "       workingday  weathersit        temp       atemp         hum   windspeed  \\\n",
       "count  731.000000  731.000000  731.000000  731.000000  731.000000  731.000000   \n",
       "mean     0.683995    1.395349    0.495385    0.474354    0.627894    0.190486   \n",
       "std      0.465233    0.544894    0.183051    0.162961    0.142429    0.077498   \n",
       "min      0.000000    1.000000    0.059130    0.079070    0.000000    0.022392   \n",
       "25%      0.000000    1.000000    0.337083    0.337842    0.520000    0.134950   \n",
       "50%      1.000000    1.000000    0.498333    0.486733    0.626667    0.180975   \n",
       "75%      1.000000    2.000000    0.655417    0.608602    0.730209    0.233214   \n",
       "max      1.000000    3.000000    0.861667    0.840896    0.972500    0.507463   \n",
       "\n",
       "            casual   registered          cnt  \n",
       "count   731.000000   731.000000   731.000000  \n",
       "mean    848.176471  3656.172367  4504.348837  \n",
       "std     686.622488  1560.256377  1937.211452  \n",
       "min       2.000000    20.000000    22.000000  \n",
       "25%     315.500000  2497.000000  3152.000000  \n",
       "50%     713.000000  3662.000000  4548.000000  \n",
       "75%    1096.000000  4776.500000  5956.000000  \n",
       "max    3410.000000  6946.000000  8714.000000  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#对数据特征采取常用统计观察分布值\n",
    "data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "season属性的不取值和出现的次数\n",
      "3    188\n",
      "2    184\n",
      "1    181\n",
      "4    178\n",
      "Name: season, dtype: int64\n",
      "\n",
      "mnth属性的不取值和出现的次数\n",
      "12    62\n",
      "10    62\n",
      "8     62\n",
      "7     62\n",
      "5     62\n",
      "3     62\n",
      "1     62\n",
      "11    60\n",
      "9     60\n",
      "6     60\n",
      "4     60\n",
      "2     57\n",
      "Name: mnth, dtype: int64\n",
      "\n",
      "weathersit属性的不取值和出现的次数\n",
      "1    463\n",
      "2    247\n",
      "3     21\n",
      "Name: weathersit, dtype: int64\n",
      "\n",
      "weekday属性的不取值和出现的次数\n",
      "6    105\n",
      "1    105\n",
      "0    105\n",
      "5    104\n",
      "4    104\n",
      "3    104\n",
      "2    104\n",
      "Name: weekday, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "#对类别型特征，观察其取值范围\n",
    "categorical_features = ['season','mnth','weathersit','weekday']\n",
    "for col in categorical_features:\n",
    "    print (\"\\n%s属性的不取值和出现的次数\"%col)\n",
    "    print (data[col].value_counts())\n",
    "    data[col] = data[col].astype('object')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4特征处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>mnth_6</th>\n",
       "      <th>...</th>\n",
       "      <th>weathersit_1</th>\n",
       "      <th>weathersit_2</th>\n",
       "      <th>weathersit_3</th>\n",
       "      <th>weekday_0</th>\n",
       "      <th>weekday_1</th>\n",
       "      <th>weekday_2</th>\n",
       "      <th>weekday_3</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  mnth_4  \\\n",
       "0         1         0         0         0       1       0       0       0   \n",
       "1         1         0         0         0       1       0       0       0   \n",
       "2         1         0         0         0       1       0       0       0   \n",
       "3         1         0         0         0       1       0       0       0   \n",
       "4         1         0         0         0       1       0       0       0   \n",
       "\n",
       "   mnth_5  mnth_6    ...      weathersit_1  weathersit_2  weathersit_3  \\\n",
       "0       0       0    ...                 0             1             0   \n",
       "1       0       0    ...                 0             1             0   \n",
       "2       0       0    ...                 1             0             0   \n",
       "3       0       0    ...                 1             0             0   \n",
       "4       0       0    ...                 1             0             0   \n",
       "\n",
       "   weekday_0  weekday_1  weekday_2  weekday_3  weekday_4  weekday_5  weekday_6  \n",
       "0          0          0          0          0          0          0          1  \n",
       "1          1          0          0          0          0          0          0  \n",
       "2          0          1          0          0          0          0          0  \n",
       "3          0          0          1          0          0          0          0  \n",
       "4          0          0          0          1          0          0          0  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#对四个类别进行 one-hot处理 （四个类别取值不多）\n",
    "x_train_cat = data[categorical_features]\n",
    "x_train_cat = pd.get_dummies(x_train_cat)\n",
    "x_train_cat.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       temp     atemp       hum  windspeed\n",
       "0  0.355170  0.373517  0.828620   0.284606\n",
       "1  0.379232  0.360541  0.715771   0.466215\n",
       "2  0.171000  0.144830  0.449638   0.465740\n",
       "3  0.175530  0.174649  0.607131   0.284297\n",
       "4  0.209120  0.197158  0.449313   0.339143"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#数值型变量预处理 标准化为0~1之间\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "mn_x = MinMaxScaler()\n",
    "numerical_features = ['temp','atemp','hum','windspeed']\n",
    "temp = mn_x.fit_transform(data[numerical_features])\n",
    "\n",
    "x_train_num = pd.DataFrame(data=temp, columns=numerical_features, index =data.index)\n",
    "x_train_num.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>mnth_6</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_3</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 32 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  mnth_4  \\\n",
       "0         1         0         0         0       1       0       0       0   \n",
       "1         1         0         0         0       1       0       0       0   \n",
       "2         1         0         0         0       1       0       0       0   \n",
       "3         1         0         0         0       1       0       0       0   \n",
       "4         1         0         0         0       1       0       0       0   \n",
       "\n",
       "   mnth_5  mnth_6     ...      weekday_3  weekday_4  weekday_5  weekday_6  \\\n",
       "0       0       0     ...              0          0          0          1   \n",
       "1       0       0     ...              0          0          0          0   \n",
       "2       0       0     ...              0          0          0          0   \n",
       "3       0       0     ...              0          0          0          0   \n",
       "4       0       0     ...              1          0          0          0   \n",
       "\n",
       "       temp     atemp       hum  windspeed  holiday  workingday  \n",
       "0  0.355170  0.373517  0.828620   0.284606        0           0  \n",
       "1  0.379232  0.360541  0.715771   0.466215        0           0  \n",
       "2  0.171000  0.144830  0.449638   0.465740        0           1  \n",
       "3  0.175530  0.174649  0.607131   0.284297        0           1  \n",
       "4  0.209120  0.197158  0.449313   0.339143        0           1  \n",
       "\n",
       "[5 rows x 32 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Join categorical and numerical features\n",
    "x_train = pd.concat([x_train_cat, x_train_num, data['holiday'],  data['workingday']], axis = 1, ignore_index=False)\n",
    "x_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "0        1         1         0         0         0       1       0       0   \n",
       "1        2         1         0         0         0       1       0       0   \n",
       "2        3         1         0         0         0       1       0       0   \n",
       "3        4         1         0         0         0       1       0       0   \n",
       "4        5         1         0         0         0       1       0       0   \n",
       "\n",
       "   mnth_4  mnth_5  ...   weekday_5  weekday_6      temp     atemp       hum  \\\n",
       "0       0       0  ...           0          1  0.355170  0.373517  0.828620   \n",
       "1       0       0  ...           0          0  0.379232  0.360541  0.715771   \n",
       "2       0       0  ...           0          0  0.171000  0.144830  0.449638   \n",
       "3       0       0  ...           0          0  0.175530  0.174649  0.607131   \n",
       "4       0       0  ...           0          0  0.209120  0.197158  0.449313   \n",
       "\n",
       "   windspeed  holiday  workingday  yr   cnt  \n",
       "0   0.284606        0           0   0   985  \n",
       "1   0.466215        0           0   0   801  \n",
       "2   0.465740        0           1   0  1349  \n",
       "3   0.284297        0           1   0  1562  \n",
       "4   0.339143        0           1   0  1600  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FE_train = pd.concat([data['instant'], x_train,  data['yr'],data['cnt']], axis = 1)\n",
    "FE_train.to_csv('FE_day.csv', index=False)\n",
    "FE_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4对特征数据集进行回归分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1导入工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "#导入模型 最小二乘线性回归、岭回归  Lasso\n",
    "from  sklearn.linear_model  import  LinearRegression,RidgeCV,LassoCV\n",
    "\n",
    "#模型评估\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import r2_score\n",
    "\n",
    "#绘图\n",
    "import matplotlib.pyplot as plt\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2加载数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "0        1         1         0         0         0       1       0       0   \n",
       "1        2         1         0         0         0       1       0       0   \n",
       "2        3         1         0         0         0       1       0       0   \n",
       "3        4         1         0         0         0       1       0       0   \n",
       "4        5         1         0         0         0       1       0       0   \n",
       "\n",
       "   mnth_4  mnth_5  ...   weekday_5  weekday_6      temp     atemp       hum  \\\n",
       "0       0       0  ...           0          1  0.355170  0.373517  0.828620   \n",
       "1       0       0  ...           0          0  0.379232  0.360541  0.715771   \n",
       "2       0       0  ...           0          0  0.171000  0.144830  0.449638   \n",
       "3       0       0  ...           0          0  0.175530  0.174649  0.607131   \n",
       "4       0       0  ...           0          0  0.209120  0.197158  0.449313   \n",
       "\n",
       "   windspeed  holiday  workingday  yr   cnt  \n",
       "0   0.284606        0           0   0   985  \n",
       "1   0.466215        0           0   0   801  \n",
       "2   0.465740        0           1   0  1349  \n",
       "3   0.284297        0           1   0  1562  \n",
       "4   0.339143        0           1   0  1600  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#导入数据\n",
    "FE_data = pd.read_csv(\"FE_day.csv\")\n",
    "#查看数据前5行\n",
    "FE_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3选取训练数据以及测试数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(731, 35)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#数据特征维数 样本数\n",
    "FE_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3.1数据准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(584, 34)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 从数据中分离输入特征x和输出y\n",
    "y = FE_data['cnt'].values\n",
    "X = FE_data.drop('cnt', axis = 1)\n",
    "\n",
    "#用于后续显示权重系数对应的特征\n",
    "columns = X.columns\n",
    "\n",
    "#将数据分割训练数据与测试数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样20%的数据构建测试样本，其余作为训练样本\n",
    "X_train, X_test, y1_train, y1_test = train_test_split(X, y, random_state=33, test_size=0.2)\n",
    "\n",
    "#训练数据数目 特征维数\n",
    "X_train.shape\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>630</th>\n",
       "      <td>631</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.736253</td>\n",
       "      <td>0.697558</td>\n",
       "      <td>0.664953</td>\n",
       "      <td>0.538460</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192</th>\n",
       "      <td>193</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.915892</td>\n",
       "      <td>0.866609</td>\n",
       "      <td>0.574979</td>\n",
       "      <td>0.366672</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>275</th>\n",
       "      <td>276</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.405012</td>\n",
       "      <td>0.410824</td>\n",
       "      <td>0.782348</td>\n",
       "      <td>0.125660</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>367</th>\n",
       "      <td>368</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.113228</td>\n",
       "      <td>0.061963</td>\n",
       "      <td>0.453728</td>\n",
       "      <td>0.707688</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>296</th>\n",
       "      <td>297</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.503656</td>\n",
       "      <td>0.496173</td>\n",
       "      <td>0.793916</td>\n",
       "      <td>0.198734</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "630      631         0         0         1         0       0       0       0   \n",
       "192      193         0         0         1         0       0       0       0   \n",
       "275      276         0         0         0         1       0       0       0   \n",
       "367      368         1         0         0         0       1       0       0   \n",
       "296      297         0         0         0         1       0       0       0   \n",
       "\n",
       "     mnth_4  mnth_5 ...  weekday_4  weekday_5  weekday_6      temp     atemp  \\\n",
       "630       0       0 ...          0          0          1  0.736253  0.697558   \n",
       "192       0       0 ...          0          0          0  0.915892  0.866609   \n",
       "275       0       0 ...          0          0          0  0.405012  0.410824   \n",
       "367       0       0 ...          0          0          0  0.113228  0.061963   \n",
       "296       0       0 ...          0          0          0  0.503656  0.496173   \n",
       "\n",
       "          hum  windspeed  holiday  workingday  yr  \n",
       "630  0.664953   0.538460        0           0   1  \n",
       "192  0.574979   0.366672        0           1   0  \n",
       "275  0.782348   0.125660        0           1   0  \n",
       "367  0.453728   0.707688        0           1   1  \n",
       "296  0.793916   0.198734        0           1   0  \n",
       "\n",
       "[5 rows x 34 columns]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看训练数据\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(147, 34)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#测试数据数目 特征维数\n",
    "X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>607</th>\n",
       "      <td>608</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.806862</td>\n",
       "      <td>0.753071</td>\n",
       "      <td>0.607113</td>\n",
       "      <td>0.112818</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>266</th>\n",
       "      <td>267</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.682257</td>\n",
       "      <td>0.637077</td>\n",
       "      <td>0.886889</td>\n",
       "      <td>0.115430</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>495</th>\n",
       "      <td>496</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.556613</td>\n",
       "      <td>0.541742</td>\n",
       "      <td>0.567695</td>\n",
       "      <td>0.601296</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>413</th>\n",
       "      <td>414</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.358285</td>\n",
       "      <td>0.362754</td>\n",
       "      <td>0.549700</td>\n",
       "      <td>0.347449</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>374</th>\n",
       "      <td>375</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.310971</td>\n",
       "      <td>0.314713</td>\n",
       "      <td>0.664804</td>\n",
       "      <td>0.340487</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "607      608         0         0         1         0       0       0       0   \n",
       "266      267         0         0         0         1       0       0       0   \n",
       "495      496         0         1         0         0       0       0       0   \n",
       "413      414         1         0         0         0       0       1       0   \n",
       "374      375         1         0         0         0       1       0       0   \n",
       "\n",
       "     mnth_4  mnth_5 ...  weekday_4  weekday_5  weekday_6      temp     atemp  \\\n",
       "607       0       0 ...          1          0          0  0.806862  0.753071   \n",
       "266       0       0 ...          0          0          1  0.682257  0.637077   \n",
       "495       0       1 ...          1          0          0  0.556613  0.541742   \n",
       "413       0       0 ...          0          0          1  0.358285  0.362754   \n",
       "374       0       0 ...          0          0          0  0.310971  0.314713   \n",
       "\n",
       "          hum  windspeed  holiday  workingday  yr  \n",
       "607  0.607113   0.112818        0           1   1  \n",
       "266  0.886889   0.115430        0           0   0  \n",
       "495  0.567695   0.601296        0           1   1  \n",
       "413  0.549700   0.347449        0           0   1  \n",
       "374  0.664804   0.340487        0           1   1  \n",
       "\n",
       "[5 rows x 34 columns]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看测试数据\n",
    "X_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "#去训练数据和测试数据的记录号\n",
    "X1_train = X_train.drop(['instant'],axis = 1)\n",
    "X1_test = X_test.drop('instant',axis = 1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>mnth_6</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>630</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.736253</td>\n",
       "      <td>0.697558</td>\n",
       "      <td>0.664953</td>\n",
       "      <td>0.538460</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.915892</td>\n",
       "      <td>0.866609</td>\n",
       "      <td>0.574979</td>\n",
       "      <td>0.366672</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>275</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.405012</td>\n",
       "      <td>0.410824</td>\n",
       "      <td>0.782348</td>\n",
       "      <td>0.125660</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>367</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.113228</td>\n",
       "      <td>0.061963</td>\n",
       "      <td>0.453728</td>\n",
       "      <td>0.707688</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>296</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.503656</td>\n",
       "      <td>0.496173</td>\n",
       "      <td>0.793916</td>\n",
       "      <td>0.198734</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  mnth_4  \\\n",
       "630         0         0         1         0       0       0       0       0   \n",
       "192         0         0         1         0       0       0       0       0   \n",
       "275         0         0         0         1       0       0       0       0   \n",
       "367         1         0         0         0       1       0       0       0   \n",
       "296         0         0         0         1       0       0       0       0   \n",
       "\n",
       "     mnth_5  mnth_6 ...  weekday_4  weekday_5  weekday_6      temp     atemp  \\\n",
       "630       0       0 ...          0          0          1  0.736253  0.697558   \n",
       "192       0       0 ...          0          0          0  0.915892  0.866609   \n",
       "275       0       0 ...          0          0          0  0.405012  0.410824   \n",
       "367       0       0 ...          0          0          0  0.113228  0.061963   \n",
       "296       0       0 ...          0          0          0  0.503656  0.496173   \n",
       "\n",
       "          hum  windspeed  holiday  workingday  yr  \n",
       "630  0.664953   0.538460        0           0   1  \n",
       "192  0.574979   0.366672        0           1   0  \n",
       "275  0.782348   0.125660        0           1   0  \n",
       "367  0.453728   0.707688        0           1   1  \n",
       "296  0.793916   0.198734        0           1   0  \n",
       "\n",
       "[5 rows x 33 columns]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看训练数据\n",
    "X1_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>mnth_6</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>607</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.806862</td>\n",
       "      <td>0.753071</td>\n",
       "      <td>0.607113</td>\n",
       "      <td>0.112818</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>266</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.682257</td>\n",
       "      <td>0.637077</td>\n",
       "      <td>0.886889</td>\n",
       "      <td>0.115430</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>495</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.556613</td>\n",
       "      <td>0.541742</td>\n",
       "      <td>0.567695</td>\n",
       "      <td>0.601296</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>413</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.358285</td>\n",
       "      <td>0.362754</td>\n",
       "      <td>0.549700</td>\n",
       "      <td>0.347449</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>374</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.310971</td>\n",
       "      <td>0.314713</td>\n",
       "      <td>0.664804</td>\n",
       "      <td>0.340487</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  mnth_4  \\\n",
       "607         0         0         1         0       0       0       0       0   \n",
       "266         0         0         0         1       0       0       0       0   \n",
       "495         0         1         0         0       0       0       0       0   \n",
       "413         1         0         0         0       0       1       0       0   \n",
       "374         1         0         0         0       1       0       0       0   \n",
       "\n",
       "     mnth_5  mnth_6 ...  weekday_4  weekday_5  weekday_6      temp     atemp  \\\n",
       "607       0       0 ...          1          0          0  0.806862  0.753071   \n",
       "266       0       0 ...          0          0          1  0.682257  0.637077   \n",
       "495       1       0 ...          1          0          0  0.556613  0.541742   \n",
       "413       0       0 ...          0          0          1  0.358285  0.362754   \n",
       "374       0       0 ...          0          0          0  0.310971  0.314713   \n",
       "\n",
       "          hum  windspeed  holiday  workingday  yr  \n",
       "607  0.607113   0.112818        0           1   1  \n",
       "266  0.886889   0.115430        0           0   0  \n",
       "495  0.567695   0.601296        0           1   1  \n",
       "413  0.549700   0.347449        0           0   1  \n",
       "374  0.664804   0.340487        0           1   1  \n",
       "\n",
       "[5 rows x 33 columns]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看测试数据\n",
    "X1_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3.2数据标准化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean_y_train: 4522.981164383562\n",
      "std_y_train: 1928.40653787788\n",
      "mean_diff: -0.048047251423073395\n"
     ]
    }
   ],
   "source": [
    "# 数据标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "mean_y_train = y1_train.mean()#原始数据均值\n",
    "print 'mean_y_train:',mean_y_train\n",
    "\n",
    "std_y_train=y1_train.std()#原始数据标准差\n",
    "print 'std_y_train:',std_y_train\n",
    "      \n",
    "y1_train=(y1_train-mean_y_train)/std_y_train#标准化后的y1_train\n",
    "y1_test=(y1_test-mean_y_train)/std_y_train#标准化后的y1_test \n",
    "\n",
    "#标准化y_test 的均值\n",
    "\n",
    "mean_y_test = y1_test.mean()\n",
    "mean_diff = mean_y_test\n",
    "print 'mean_diff:',mean_diff"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5模型训练和评估"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.1最小二乘模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 线性回归\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# 使用默认配置初始化\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 训练模型参数\n",
    "lr.fit(X1_train, y1_train)\n",
    "\n",
    "# 预测\n",
    "y_train_pred_lr = lr.predict(X1_train)\n",
    "y_test_pred_lr = lr.predict(X1_test)\n",
    "y_test_pred_lr+=mean_diff\n",
    "\n",
    "\n",
    "#用于后续显示权重系数对应的特征\n",
    "columns = X1_train.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rmse_train: 0.3863713091528291\n",
      "rmse_test: 0.42894489715377704\n",
      "('r2_score_train:', 0.850717211463529)\n",
      "('r2_score_test:', 0.8225377569799628)\n"
     ]
    }
   ],
   "source": [
    "#RMSR性能\n",
    "rmse_train = np.sqrt(mean_squared_error(y1_train,y_train_pred_lr))\n",
    "rmse_test = np.sqrt(mean_squared_error(y1_test,y_test_pred_lr))\n",
    "print 'rmse_train:',rmse_train\n",
    "print 'rmse_test:',rmse_test\n",
    "\n",
    "#r2_score性能测试\n",
    "r2_score_train=r2_score(y1_train,y_train_pred_lr)#训练集拟合度\n",
    "r2_score_test=r2_score(y1_test,y_test_pred_lr)#测试集拟合度\n",
    "print(\"r2_score_train:\",r2_score_train)\n",
    "print(\"r2_score_test:\",r2_score_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评价：训练集要好点，测试集稍差"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2岭回归模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha: 1.0\n",
      "cv of rmse: 0.4086009868705135\n",
      "rmse_train: 0.38699977782879325\n",
      "rmse_test: 0.4266888489288939\n",
      "r2_score_train: 0.8502311719604646\n",
      "r2_score_test: 0.8243995836134212\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import  RidgeCV\n",
    "from sklearn.metrics import r2_score\n",
    "\n",
    "#设置超参数（正则参数）范围\n",
    "alphas = [ 0.01, 0.1, 1, 10,100,1000]\n",
    "\n",
    "#生成一个RidgeCV实例\n",
    "ridge = RidgeCV(alphas=alphas, store_cv_values=True)  \n",
    "\n",
    "#模型训练\n",
    "ridge.fit(X1_train, y1_train)    \n",
    "\n",
    "#交叉验证得到最佳超参数\n",
    "alpha = ridge.alpha_\n",
    "print 'alpha:',alpha\n",
    "\n",
    "#交叉验证估计的测试误差\n",
    "mse_cv = np.mean(ridge.cv_values_,axis=0)\n",
    "rmse_cv = np.sqrt(mse_cv)\n",
    "print 'cv of rmse:',min(rmse_cv)\n",
    "\n",
    "\n",
    "#预测\n",
    "y_train_pred_ridge = ridge.predict(X1_train)\n",
    "y_test_pred_ridge = ridge.predict(X1_test)\n",
    "y_test_pred_ridge += mean_diff\n",
    "\n",
    "#测试误差\n",
    "rmse_train = np.sqrt(mean_squared_error(y1_train,y_train_pred_ridge))\n",
    "rmse_test = np.sqrt(mean_squared_error(y1_test,y_test_pred_ridge))\n",
    "\n",
    "print 'rmse_train:',rmse_train\n",
    "print 'rmse_test:',rmse_test\n",
    "\n",
    "r2_score_train = r2_score(y1_train,y_train_pred_ridge)\n",
    "r2_score_test = r2_score(y1_test,y_test_pred_ridge)\n",
    "\n",
    "print 'r2_score_train:',r2_score_train\n",
    "print 'r2_score_test:',r2_score_test\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEICAYAAAD7pTujAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcHVWZ//HPV2QPEiBRQZZIAKMQCBCCDKBsggLDojBsKgGFQVTGn8MmMpgRUTQ6iCIiILJlkGEdkCUgCAkxLAGyIRCWhAFBSFAQBGIkz++Pcy6p3HT3vbf7rt3f9+vVr65bdarqqUqnT59Tp56jiMDMzKydvKvVAZiZmZVz5WRmZm3HlZOZmbUdV05mZtZ2XDmZmVnbceVkZmZtx5WTdTRJH5L0sKTXJB0naWVJN0p6VdJVkg6TdFsVxzlF0oXNiLmHGNaX9Lqk5ep0vHGSLq/Hsao4V4/3WdJdkr7YjFjqoZZ4JYWkjRod00DjysmaQtKhkqblX74vSLpF0g51OPSJwF0RsVpE/AQ4AHgfsFZEHBgREyJi90oHiYjvRkSff3lKGpZ/Wb271n0j4v8iYlBEvN2L8+4k6bla96vh+BdL+nv+9/uzpNsljShtr/Y+NyCucfl+H1e2/mt5/bhmx2T14crJGk7S14EfA98lVRzrA+cC+9bh8BsAj5R9nhMR/6jDsW1pP4iIQcAHgD8Cv2xxPCVzgMPL1n0+r7cO5crJGkrS6sC3gS9HxLUR8beIWBQRN0bECbnMipJ+LOn5/PVjSSsWjrG3pOmSXpH0e0mb5/V3AjsD5+S/6K8ATgMOyp+/IGmspHsKx9o0/9X/Z0kvSjolr1+qC0zSR/O5XpE0Q9JOhW13STpd0pTcnXibpCF586T8/ZUcw3aSNpJ0d+5qXCDpym7u1VKtrgrnKe63KnALsE4+5+uS1smbV5B0ad7/EUmjC/utI+kaSfMlzS1vfXQnIt4E/gcYVThW+X3+hKTH8jWfA6iwbTlJP8r3Yq6kr5Rd9+qSfplb2H+U9J0KXZ0PAKtI2jTvvymwcl5fvE9HSXoy/9vfULhHPcabtx8p6VFJf5E0UdIG1dwr6z1XTtZo2wErAdf1UOabwEdJv+y2AMYApwJI2gq4CPhXYC3gF8ANklaMiF2AycBXcnfYIaTW2ZX581J/2UtaDfgtcCuwDrARcEd5MJI+ANwEfAdYEzgeuEbS0EKxQ4EjgPcCK+QyAB/L3wfnGKYCpwO3AWsA6wI/7eFelOvuPO+IiL8BnwKez+ccFBHP5837AL8GBgM3AOfka3wXcCMwg9QS2hX4mqQ9KgWUK8NDgCe72T4EuIb0bzgEeArYvlDkqBzvKGArYL+yQ1wC/IP077MlsDtQqcv1MlJrCVIr6tKymHYBvgf8C7A28AzpvlSMV9J+wCnAp4GhpJ+5KyrEY33kyskabS1gQYVutsOAb0fESxExH/hP4HN521HALyLivoh4OyIuARaSKrNa7Q38KSJ+FBFvRcRrEXFfF+U+C9wcETdHxOKIuB2YBuxZKPOriJjTVSuiC4tI3Y3r5PPe00PZcrWcpyv35Ot4m/QLfIu8fhtgaER8OyL+HhFPAxcAB/dwrOMlvQK8BuzAkn+jcnsCf4iIqyNiEalL90+F7f8CnB0Rz0XEX4AzSxskvY9UcX0tt7JfAs6qEBfA5cAhkpbPZcsHghwGXBQRD0XEQuAbwHaShlUR778C34uIR/PP8XeBUW49NZYrJ2u0l4Eh6nmAwDqkv2RLnsnrIP1S//fcvfZK/uW4XmF7LdYj/VVcyQbAgWXn3IH0F3dJ8ZfXG8CgHo53Iqmb6P7ctXZkDTHXcp5q9l8p/1tsQOoGLF7jKaRngt35YUQMBoYBbwIf6qbcOsCzpQ+Rsks/2932suUNgOWBFwpx/YLUcuxWRPwfqSX3XeCJiHi2rMhSP2MR8TrpZ/MDVcS7AXB2IZ4/k/49P9BTTNY3NY8oMqvRVOAtUtfN1d2UeZ6lBzasn9dB+iVxRkScUYdYniV1R1VT7rKIOKoX51gmzX9E/InUAkRphOJvJU2KiC67xXqp1ukFngXmRsTGNZ8o4v8k/RtwiaTf5FZd0QukPwQAkKTi57x93cLn4rZnSS3jIb0Y1HIpqQv4iC62lX7GSjGtSmrV/7GKeEs/gxNqjMf6wC0na6iIeJU0SOFnkvaTtIqk5SV9StIPcrErgFMlDc39/6expFvmAuAYSdsqWVXSXvn5Ua1+A7xfaZjxipJWk7RtF+UuB/5Z0h754f1KSkO11+2ibLn5wGJgw9IKSQcW9v0LqSKpebh4BS8CaykNQKnG/cBfJZ2k9G7YcpI2k7RNNTvnrs7ngaO72HwTsKmkT+dW2nHA+wvb/wf4N0kfkDQYOKlw3BdIz+d+JOk9kt4labikj1cR1pWk51P/08W2/waOkDRKabDNd4H7ImJeFfGeB3yjMOBidUkHVhGP9YErJ2u4iPgv4OukB87zSX+JfgW4Phf5DumZzkxgFvBQXkdETCO1Os4h/WJ/EhjbyzheAz4B/DOpu+sJ0mi/8nLPkoa5n1KI9wSq+P8SEW8AZwBTcjfQR0nPd+6T9DppUMK/RcTc3lxDD+d9jFTJP53P22O3Z34G9c+kZ1hzgQXAhUC1lRvAeOBEFUZW5mMvAA4kPUt6GdgYmFIocgGpApoJPAzcTBoAUaqwP08a/PEH0r/51SzdpdrdNb0ZEb/toiVHRNwB/Adp4MMLwHDyc6xK8UbEdcD3gV9L+iswm/RczBpInmzQzFpJ0qeA8yLCAwzsHW45mVlT5W7EPSW9Ow/b/xY9v2pgA5BbTmbWVJJWAe4GRpBG/d1E6ur8a0sDs7biysnMzNqOu/XMzKzt+D2nXhoyZEgMGzas1WGYmXWUBx98cEFEDK1UzpVTLw0bNoxp06a1Ogwzs44i6ZnKpdytZ2ZmbcgtJ7M6u+PO4a0Owayhdt2lmhSVfeOWUzcqzB9jZmYNNGBbTpJOJ03lcHb+fAYpP9n+pPQmo4CPtC5CM7OBayC3nH5Jnto5T7x2MClD8RjgmxGxTMUk6WhJ0yRNmz9/flODNTMbSAZs5ZSzEb8sqTTT5sOkpI/3d5eUMyLOj4jRETF66NCKIyHNzKyXBmy3XnYhKcP1+0nzwAD8rWXRmJkZ4MrpOuDbpJk3DwV2bG041h80YySTWX83oCuniPi7pN8Br0TE22kCTDMza7UBXTnlgRAfJU00RkTcBdzVwpDMzIwBPCBC0kdIs6reERFPtDoeMzNbYsC2nCLiD8CGrY7DzMyW1bGVk6TBwKERcW6rYzErGjduXKtDMHtHp/48dnK33mDg2FYHYWZm9dfJldOZwHBJ0yWNl3SCpAckzZT0nwCShkl6TNKFkmZLmiBpN0lTJD0haUwuN07SZZLuzOuPaumVmZkNcJ1cOZ0MPBURo4DbgY1JqYdGAVtL+lgutxFwNrA5MIL0PtMOwPHAKYXjbQ7sBWwHnCZpnfITOn2RmVlzdHLlVLQ7S1IQPUSqhDbO2+ZGxKyIWAw8QhqdF8AsYFjhGP8bEW9GxALgd6SKbilOX2Rm1hwdOyCijIDvRcQvllopDQMWFlYtLnxezNLXH2XHLP9sVpVOfQBt1k46ueX0GrBaXp4IHClpEICkD0h6b43H21fSSpLWAnYCHqhbpGZmVpOObTlFxMt5YMNs4Bbgv4GpOQXR68BngbdrOOT9wE3A+sDpEfF8nUM2M7MqdWzlBBARh5atOruLYpsVyo8tLM8rbgPmRMTR9YzPzMx6p5O79czMrJ/q2MpJ0imVS1UnIsZFxA/rdTwzM+ubTu7WOwX4bquDMCv33MmTWx2CdaB1z/R0ckUd0XKSdL2kByU9kl+EPRNYOWeHmJDLfFbS/XndLyQtl9e/Lun7ef/fShoj6S5JT0vaJ5cZK+l/Jd0q6XFJ32rh5ZqZDXgdUTkBR0bE1sBo4DhgPPBmRIyKiMMkfRg4CNg+Z4x4Gzgs77sqcFfe/zXgO8AngP1Js+CWjMn7jAIOlDS6CddlZmZd6JRuveMk7Z+X12NJ9oeSXYGtgQfyUPKVgZfytr8Dt+blWcDCiFgkqTxDxO0R8TKApGtJKY6mFU8i6WjgaID111+/71dlZmZdavvKSdJOwG7AdhHxhqS7gJXKiwGXRMQ3ujjEopyuCAoZIiJisaSaMkRExPnA+QCjR492BgkzswZp+8oJWB34S66YRpCmVQdYJGn5iFgE3AH8r6SzIuIlSWsCq0XEMzWc5xN5vzeB/YAj63kRNnD4wbZZ33XCM6dbgXdLmgmcDtyb158PzJQ0Ic9qeypwWy53O7B2jee5B7gMmA5cExHTKpQ3M7MG0ZIer4FL0lhgdER8pdp9Ro8eHdOmuf4yM6uFpAcjouKAs05oOZmZ2QDTCc+cGi4iLgYubnEYZmaWueVkZmZtp+ktp/x857bSlBSS5pGe9yyo83luJk3JDnBoRJxbofytpJGA90TE3vWMxQaWHx3kH596+vcrf9PqEKwFWtFyGgusU48Dlb2ntJSI2DMiXgEGA8dWcbjxwOfqEZeZmfVNxcpJ0omSjsvLZ0m6My/vKulySbtLmirpIUlXFWajPU3SA5JmSzpfyQGkFEQTcg68lfNpvpr3n5XfZULSqpIuysd4WNK+ef3YfJ4bSUPH15Y0KR9vtqQdc7l5koYAZwLD8/bx3V1nRNxBSm/U0704WtI0SdPmz59f6daZmVkvVdNymgSU3iocDQyStDwpvc8s0vtFu0XEVqR0P1/PZc+JiG0iYjNSOqG9I+LqXOawnBfvzVx2Qd7/58Dxed03gTsjYhtgZ2C8pFXztu2AwyNiF1LX3cScU28L0ntKRScDT+XznVDNTelORJwfEaMjYvTQoUP7cigzM+tBNc+cHgS2lrQaKfXPQ6RKakfgBuAjwJSc024FYGreb2dJJwKrAGsCjwA3dnOOawvn+nRe3h3YR1KpslqJNIU6pDx4f87LDwAX5Qrz+ogor5zMzKzDVKyccpLUecARwO+BmaSWzHBgLqmiOKS4j6SVgHNJAx2elTSOZfPhFS3M398uxCTgMxHxeNmxtwX+VohvkqSPAXsBl0kaHxGXVrous0bxA3yzvqt2QMQkUnfbJGAycAyp++xeYHtJGwFIWkXSJiypiBbkZ1AHFI71GrBaFeecSHoWpXzsLbsqJGkD4KWIuAD4JbBVWZFqz2dmZm2i2sppMilX3dSIeBF4C5gcEfNJo++uyDnt7gVG5FFyF5CeSV1P6noruRg4r2xARFdOB5Yn5c+bnT93ZSdguqSHgc8AZxc35mkwpuTBEt0OiJA0GbgK2FXSc5L26CE2MzNrIOfW6yXn1jMzq51z65mZWccaULn1JI0kTYtRtDAitm1FPGZm1rUBVTlFxCxgVE9lJN0AbJjfzzKr2c+OubPVIbS1L5+3S6tDsA7gbr0CSZ8GXm91HGZmA13TKqecjugmSTPyyLmDJG0t6W5JD0qaKGntXPaonLZohqRrJK2S1x+Y950haVJet5KkX+XURw9L2jmvHyvpWkm3SnpC0g8qxDeIlN3iOz2UcfoiM7MmaGbL6ZPA8xGxRe4yuxX4KXBARGwNXASckctem1MfbQE8Cnwhrz8N2COv3yev+zJARIwEDgEuyS8BQ+rCOwgYCRwkab0e4jsd+BHwRncFnL7IzKw5mlk5zQJ2k/T9nJx1PWAz4HZJ00k5+tbNZTeTNFnSLOAwYNO8fgpwsaSjgOXyuh3Igxwi4jHgGWCTvO2OiHg1It4C/gBs0FVgkkYBG0XEdfW7XDMz662mDYiIiDmStgb2BL4H3A48EhHbdVH8YmC/iJihNP/TTvkYx+T0RXuRXrwdRUpz1J2FheViaqRy25HyB87LZd4r6a6I2Km6qzNbwg/8zfqumc+c1gHeiIjLgR8C2wJDJW2Xty8vqdRCWg14ISdzPaxwjOERcV9EnAYsILW+JpXK5NRJ6wNL5eOrJCJ+HhHrRMQwUktsjismM7PWaeZQ8pGkaS8WA4uALwH/AH4iafUcy49J2cv/A7iP1EU3iyW58cZL2pjUWroDmAE8RkqHNCsfb2xELMwp+czMrAM5fVEvOX2RmVntnL7IzMw61oDKEAEg6T5gxbLVn8vZI8z67NERH251CG3rw4892uoQrEP0m5aTpMGSji183knSMrO+RcS2ecr24tcsSbtIeii/5HuJpAFXcZuZtYt+UzkBg4FjK5bqgqR3AZcAB+cXhJ8BDq9jbGZmVoO2qpwkDZP0mKQLcwtmgqTdJE3JKYjGSBon6SJJd0l6WtJxefczgeF5EsPSpIKDJF2djzmhNKtuF9YiZSefkz/fTpq4sDw+py8yM2uCtqqcso1Is9luDowADiW9e3Q8cEouMwLYAxgDfCu/D3Uy8FTupjshl9sS+BrwEWBDYPtuzrkAWF5SaQTJAaR3qJbi9EVmZs3RjpXT3IiYFRGLSe883RFpvPssYFguc1NELIyIBcBLwPu6Odb9EfFcPtb0wv5Lycc/GDhL0v3Aa6R3pszMrAXa8aF/MeXQ4sLnxSyJt9q0RNWWIyKmAjsCSNqdJfn5zGriEWlmfdeOLafeeo0lmSRqJum9+fuKwEnAeXWKy8zMatRvKqeIeBmYkgdSjK+4w7JOkPQoMBO4MSI8namZWYs4fVEvOX2RmVntnL7IzMw6VjsOiGgoSdcBHyxbfVJETGxFPNb/jLxkZKtDeMesw52VyzpT01tOksbmuZ1Kn+dJGtKA89ycUxotldYoIvYvT18EvChpqqRHJM2UdFC94zEzs+q1oltvLLBOpULV6Cn/XUTsGRGvUF1aozeAz0fEpsAngR9LGlyPGM3MrHYVKydJJ5ZSBEk6S9KdeXlXSZdL2j23Oh6SdJWkQXn7aZIeyKPnzldyADAamJDTDK2cT/PVvP8sSSPy/qvmNEUPSHpY0r55/dh8nhuB2yStLWlSPt5sSaV3lUotsq7SGi0lIuZExBN5+XnSi73LpIBw+iIzs+aopuU0ifxyKqliGZTTBe1AytpwKrBbRGwFTAO+nsueExHb5ESqKwN7R8TVucxhuUvtzVx2Qd7/56Q0RQDfBO6MiG2AnUmz4K6at20HHB4Ru5DSG03M3XNbkDJBFHWV1qhbksYAKwBPlW9z+iIzs+aoZkDEg8DWklYjZVx4iFRJ7QjcQMpbNyXnVF0BmJr321nSicAqwJqkVEQ3dnOOawvn+nRe3h3YR1KpsloJWD8v3x4Rf87LDwAX5Qrz+ogor5yqJmlt4DJSxbe4t8cxM7O+qVg5RcQiSfOAI4Dfk15S3RkYDswlVRSHFPeRtBJwLjA6Ip6VNI5UuXSnlGaomGJIwGci4vGyY28L/K0Q3yRJHwP2Ai6TND4iLq10XeUkvQe4CTg1Iu6tdX+zEo+QM+u7agdETCJ1t00CJgPHkLrP7gW2l7QRgKRVJG3CkopoQX4GdUDhWNWmGZpIehalfOwtuyokaQPgpYi4APglsFVZkYrnk7QCcB1waURcVUVsZmbWQNVWTpOBtYGpEfEi8BYwOSLmk0bfXSFpJqmyGpFHyV1AeiZ1PanrreRi4LyyARFdOR1YHpgpaXb+3JWdgOmSHibNwXR2cWOVaY3+BfgYMDbHNV3SqB5iMzOzBnL6ol5y+iIzs9o5fZGZmXWsAZW+SNJI0mi8ooURsW0r4rF+atzqLT7/q609v1kd9JuWU3maIkk7SfpNsUyeYXdU2de2ubwknSFpjqRHSy8em5lZ8/Wbyonq0hT1ZCywHmlAx4eBX9cjKDMzq11bVU6Shkl6TNKFeXTdBEm7SZoi6QlJYySNy2mN7pL0dKGF01WaokGSrs7HnFAalt6NLwHfLr18GxEvdRGf0xeZmTVBW1VO2Uak4eCbAyNI6Yl2IL1ndUouMwLYAxgDfCtnh+gqTdGWwNdIWSw2BLbv4bzDgYNy5XOLpI3LCzh9kZlZc7Rj5TQ3PxtaTEp5dEek8e6zgGG5zE0RsTAiFpCStL6vm2PdHxHP5WNNL+zflRWBt/IQxwuAi/p+KWZm1hvtOFpvYWF5ceHzYpbEWyxTTHnU07F6KgfwHHBNXr4O+FU1wZotw6PlzPqsHVtOvVVtWqTuXA/skpc/Dszpc0RmZtYr7dhy6pWIeDkPnJgN3EJK4lqLM0nzTP0/4HXgi/WO0czMquP0Rb3k9EVmZrVz+iIzM+tY/aZbr1qSrgM+WLb6pIiY2Ip4rP8ZdnKtPcr1M+/MvVp2brN66jeVk6TBwKERcW7+vBNwfETsXSwXEft3s//FpIEQpaFWY/syq66ZmfVef+rW62v6IoATCjn3XDGZmbVIW1VOLU5fVE18Tl9kZtYEbVU5Za1KXwRwhqSZks6StGL5RqcvMjNrjnZ85jQ3ImYBSHonfZGkUvqi6eT0RcBCSRXTF+VjldIX3dNN2W8AfwJWAM4HTgK+XZcrsgHFgxLM+q4dW04tSV8UES9EspCUumhMLUGbmVn9tGPl1Ft9Sl8kae38XcB+wOw6xWVmZjVqx269XqlD+qIJkoYCInUdHlPvGM3MrDpOX9RLTl9kZlY7py8yM7OO1W+69arNENFd+iLgH8B4UoX9OilDxJONjtv6n0anL/JoQBsI+k3lxJIMEef2VKiH9EVzgH0j4lFJxwKnAmPrHaSZmVXWVt16Lc4QEcB78vLqwPMNukwzM6ugHVtOGwEHAkcDD7AkQ8Q+pAwR00kZInYmDR1/XNLPSRkiNouIUfBOt96WwKakimYKKUNEdy/hfhG4WdKbwF+Bj5YXkHR0jov111+/71dqZmZdaquWUzY3ImZFxGLgnQwRQClDBOQMERGxAKiYISIfa3ph/678P2DPiFiX9BLuf5UXcPoiM7PmaMeWU9MzROT3m7aIiPvyqiuBW2uI2ewdHrBg1nft2HLqrb5kiPgLsLqkTfLnTwCP1iUqMzOrWTu2nHqlLxkiIuIfko4CrpG0mFRZHdmgUM3MrAJniOglZ4gwM6udM0SYmVnH6jfdetXqLkNERExsRTxmZrasflM5VZu+qIcMEZNZMqDivaRh6Ps1LmJrB+//3fS6H/NPO4+q+zHNBpr+1K1XSl/UKxGxY57ifRQwFbi2bpGZmVlN2qpyanH6olIMqwG7ANc36DLNzKyCduzWa1X6opL9SVkp/lq+wemLzMyao61aTlmr0heVHAJc0dUGpy8yM2uOdmw5NT19UYmktYAxpNaTDQAevGDWntqx5dRbfUlfVHIg8JuIeKsO8ZiZWS/1m8opIl4GpuSBFOMr7tC1g+mmS8/MzJrH6Yt6yemLzMxq5/RFZmbWsdqucsrvOs2uofzFkg7IyxdK+kgXZcZKOicvX5ffhSp+7VG/KzAzs75qx9F6vRYRX6yijEfiGXfcObxhx951l6cadmyzgaLtWk7ZcpIukPSIpNskrSxplKR7Jc3MrZ81ynfKWSNG5+UjJM2RdDfp5dtSmX+WdJ+khyX9VtL7JL0rZ6AYmsu8S9KTkoY07YrNzOwd7Vo5bQz8LCI2BV4BPgNcSsoevjnphdxvdbezpLWB/yRVSp8Ail199wAfjYgtgV8DJ+aXdC8HDstldgNm5Jd8i8c9WtI0SdPmz59fh8s0M7OutGvlNDciSumiHwSGA4Mj4u687hLgYz3svy1wV0TMj4i/A1cWtq0LTJQ0CziBlN4I4CLg83n5SOBX5Qd1hggzs+Zo18qpPLPD4F4co7sx8j8FzomIkcC/AisBRMSzwIuSdiFVbrf04pxmZlYHnTIg4lXgL5J2jIjJwOeAu3sofx9wdk5H9FdS5ocZedvqwB/z8uFl+11I6t67LCLerlfw1n48aMGsvXVK5QSpIjlP0irA08AR3RWMiBckjSPNy/QC8BCwXN48DrhK0h+Be1l6VtwbSN15y3TpmZlZ8zhDREEe6XdWROxYqawzRJiZ1a7aDBGd1HJqKEknA19iyYg9MzNrkXYdENF0EXFmRGwQEZUmIzQzswZz5WRmZm2n33TrSRoMHBoR5+bPOwHHR8TeVe4/ARgNLALuB/41IhY1KFxrknHjxg2Ic5r1N/2p5TQYOLYP+08ARgAjgZWBinn6zMysMdqqcsoZyR/L2cVnS5ogaTdJU3LuuzGSxkm6KOfRe1rScXn3M4HhOct4abLBQZKuzsecIEndnTsibo6M1HJat4v4nL7IzKwJ2qpyyjYCzgY2J7VkDgV2AI4HTsllRgB7AGOAb0laHjgZeCoiRkXECbnclsDXSLn1NqSQALY7+VifA24t3+b0RWZmzdGOldPciJiVk7E+AtyRWzOzgGG5zE0RsTAnZn0JeF83x7o/Ip7Lx5pe2L8n5wKTciYKMzNrgXYcEFHMq7e48HkxS+Itz73X3XVUWw4ASd8ChpJy7lk/4MEJZp2pHSun3noNWK23O0v6IqmrcNfc0jIzsxZpx269XomIl4EpeSDF+Io7LOs8Uvfg1Dyo4rT6RmhmZtVybr1ecm49M7PaVZtbr9+0nMzMrP/oT8+cqiLpOpaeJgPS9O8TWxGP9c1zJ7ffoMp1z6yY1N7MKug3LSdJgyUdW/i8k6TflJeLiP3zu1DFr4mSviLpSUkhaUhzozczs6J+UznR9/RFU4DdgGfqE46ZmfVWW1VOLU5f9HBEzKsQn9MXmZk1QVtVTllL0xf1xOmLzMyaox0rp1anLzIzsxZrx9F6LUtfZJ3HI+PM+qd2bDn1Vp/SF5mZWfvoN5VTX9MXSTpO0nOkeZxmSrqw7kGamVlVnL6ol5y+yMysdk5fZGZmHWvADRBw+qL29aOD9m51CHXx71cuk5jEzGrUUS2nalMU9bD/V4CRwBbAbqX0RcAzkqZKWijp+PpHbmZmteioyonGpSj6M3Ac8MM+HNvMzOqk6ZVTO6YoioiXIuIBYFGF2J2+yMysCVrVcmrbFEU9cfoiM7PmaFXl5BRFZmbWrVaN1nOKIluGR7mZWUmnDYhwiiIzswGgoyqnRqUokvT+vP7rwKmSnpP0nroGb2ZmVXP6ol5y+iIzs9o5fZGZmXWsfjl4wCmK2t/Pjrmz1SE0zJfP26XVIZh1vI5qOVWbvigi9i+lJip8TZT0FUlPSgowLmxCAAAO/0lEQVRJQwrHOUzSzPz1e0lbNOuazMxsWR1VOdG49EVzgY9HxObA6cD5fTiHmZn1kdMXpfW/j4i/5I/3kkbzdRW70xeZmTWB0xct6wvALV1tcPoiM7PmcPqiAkk7kyqnk3p7DDMz6zunL8okbQ5cCHwqv+xrDeQRbWbWk04bENGQ9EWS1geuBT4XEXPqfXwzM6tNR1VOjUpfBJwGrAWcmwdbOPWDmVkLOX1RLzl9kZlZ7Zy+yMzMOpbTFy3Z5wzg88AaETGokfF1skdHfLjVIbS9Dz/2aKtDMOt4/bJyioj9e7HbjcA5wBN1DsfMzGrUtG49SatKuknSjDyg4SBJW0u6W9KDkiZKWjuXPUrSA7nsNZJWyesPzPvOkDQpr1tJ0q8kzZL0cH5XCUljJV0r6daceeIHPcUXEfdGxAuNvg9mZlZZM585fRJ4PiK2iIjNgFuBnwIHRMTWwEXAGbnstRGxTURsATxKejEW0qi6PfL6ffK6LwNExEjgEOASSSvlbaOAg4CRwEGS1uvLBTh9kZlZczSzcpoF7Cbp+5J2BNYDNgNulzQdOJUlOe02kzRZ0izgMGDTvH4KcLGko4Dl8rodgMsAIuIxUlLXTfK2OyLi1Yh4C/gDsEFfLsDpi8zMmqNpz5wiYo6krYE9ge8BtwOPRMR2XRS/GNgvImZIGgvslI9xjKRtgb2A6ZJGAd0meqVO2SNsCT/sN7NmaOYzp3WANyLicuCHwLbAUEnb5e3LSyq1kFYDXsjJXg8rHGN4RNwXEacBC0itr0mlMpI2AdYHHm/SZZmZWQM0syUxEhgvaTGwCPgS8A/gJ5JWz7H8mJQI9j+A+0hddLNYkrJovKSNSa2lO4AZwGPAebkL8B/A2IhY2MPMGV3KAyYOBVbJWSQujIhxvb9cMzPrLWeI6CVniDAzq50zRJiZWcdq5jOnmyUNrqH8MEmzGxDHfTm5a/FrZFmZ1+t9XjMzq14zR+vt2axz9SQitm11DI0w8pKRlQtZU8w6fFarQzDreHVrOUk6UdJxefksSXfm5V0lXS5pnqQhuUX0qKQLJD0i6TZJK+eyW+fsD1PJL9fm9ZtKuj+3cmZK2jgf5zFJl+R1VxcySXSXeWJ4zhjxYH6PakRe/0FJU3NWitPrdU/MzKx36tmtNwnYMS+PBgbloeA7AJPLym4M/CwiNgVeAT6T1/8KOK6Ld5+OAc6OiFH52M/l9R8Czo+IzYG/Asfmc3aXeeJ84Kt5/fHAuXn92cDPI2Ib4E+9vQFmZlYf9aycHgS2lrQa6eXXqaSKZEeWrZzmRsT0wn7D8nDywRFxd15/WaH8VOAUSScBG0TEm3n9sxExJS9fTqoIP0QXmSckDQL+Cbgqr/8FsHbed3vgii7OuxSnLzIza466PXOKiEWS5gFHAL8HZgI7A8NJ+fGKyjM3rEx6d6nLce0R8d+S7iNlhpgo6YvA012Uj3ycZTJPSHoP8EpufXV5mh4vMMVxPqn1xejRoz0G38ysQeo9IGISqbvsSNLLs/8FPBgRUeml2Ih4RdKrknaIiHtYOjPEhsDTEfGTvLw5qXJaX9J2ETGVlPT1HlJ2iKGl9bmbb5OIeETSXEkHRsRVSgFtHhEzSDn7Dia1vg6jA/khvJn1J/UeSj6Z1FU2NSJeBN5i2S69nhwB/CwPiHizsP4gYHbujhsBXJrXPwocLmkmsCbpudHfgQOA70uaAUwndedBqni+kNc/Auyb1/8b8GVJDwCr13LBZmZWfx2bIULSMOA3efqNpnOGCDOz2jlDhJmZdayOnUIiIuaRRuWZmVk/029aTo1Kd2RmZs3XsS2nAW2cx2y0tXGvtjoCs47Xb1pO2XLlaZEk3SVpNEBOnzQvL4+VdL2kG/MQ869I+rqkhyXdK2nNll6JmdkA1t8qp+7SInVnM9IEg2NIKY7eiIgtSRkpPl9e2BkizMyao79VTsukRapQ/ncR8VpEzAdeBW7M62d1tW9EnB8RoyNi9NChQ+sUspmZletvlVN5WqR3k6ZuL13nSj2UX1z4vBg/jzMza5mB8At4HrA1cD8pc0Tn8wN3M+vn+lvLqSs/BL4k6ffAkFYHY2ZmlXVs+qJWc/oiM7PaOX2RmZl1LFdOZmbWdlw5mZlZ22n6aD1JY4HbIuL5/HkeMDoiFtT5PDeTXrAFODQizu2h7AbAtcBywPLATyPivHrGU27YyTc18vDWQvPO3KvVIZh1vFa0nMYC69TjQJK6rVwjYs+IeAUYDBxb4VAvAP+Up3DfFjhZUl1iNDOz2lWsnCSdKOm4vHyWpDvz8q6SLpe0u6Spkh6SdJWkQXn7aZIekDRb0vlKDgBGAxMkTZe0cj7NV/P+sySNyPuvKumifIyHJe2b14/N57kRuE3S2pIm5ePNlrRjLjdP0hDgTGB43j6+q2uMiL9HROkF3BW7uy9OX2Rm1hzVtJwmATvm5dHAIEnLAzuQ0vycCuwWEVsB04Cv57LnRMQ2eabalYG9I+LqXOawiBgVEaWp2Bfk/X8OHJ/XfRO4MyK2AXYGxktaNW/bDjg8InYhdd1NzK2eLUjTshedDDyVz3dCdxcpab083fuzwPdL3Y5FTl9kZtYc1VRODwJbS1qNlN5nKqmS2hF4E/gIMEXSdOBwYIO8386S7pM0C9gF2LSHc1xbONewvLw7qXttOnAXKfXQ+nnb7RHx57z8AHCEpHHAyIh4rYprWkZEPBsRmwMbAYdLel9vjmNmZn1XcUBERCzKgxaOAH4PzCS1ZIYDc0kVxSHFfSStBJxLGujwbK44yvPaFZW61Er58AAEfCYiHi879rbA3wrxTZL0MWAv4DJJ4yPi0krX1Z2IeF7SI6TK9+reHqcSPzQ3M+tetQMiJpG62yYBk4FjSN1n9wLbS9oIQNIqkjZhSUW0ID+DKua0ew1YrYpzTiQ9i1I+9pZdFcoj7V6KiAuAXwJblRWpeD5J65aef0laA9geeLynfczMrHGqrZwmA2sDUyPiReAtYHKeamIscEV+XnMvMCKPkruA9EzqelLXW8nFwHllAyK6cjppWPfMPP366d2U2wmYLulh0vxNZxc3RsTLpG7H2d0NiAA+DNwnaQZwN/DDiJjVQ2xmZtZAzq3XS5LmA8/kj0OAur6n1WCOt7Ecb2M53sZrZMwbRETFEWWunOpA0rRqEhm2C8fbWI63sRxv47VDzANhPqd3SBoJXFa2emFEbNuKeMzMrGsDqnLKz5FGtToOMzPrmRO/1sf5rQ6gRo63sRxvYznexmt5zH7mZGZmbcctJzMzazuunMzMrO24cqqCpDUl3S7pifx9jS7K7JxfLC59vSVpv7ztYklzC9saPiijmphzubcLcd1QWP/BnBvxCUlXSlqh1fFKGpUz4D8iaaakgwrbmnKPJX1S0uOSnpR0chfbV8z368l8/4YVtn0jr39c0h6NiK8X8X5d0h/y/bwjZ1wpbevyZ6PF8Y6VNL8Q1xcL2w7PPz9PSDq8TeI9qxDrHEmvFLa14v5eJOmlnNigq+2S9JN8PTMlbVXY1tz7GxH+qvAF/AA4OS+fTMpa3lP5NYE/A6vkzxcDB7RjzMDr3az/H+DgvHwe8KVWxwtsAmycl9chzcM1uFn3mDQZ5VPAhsAKwAzgI2VljgXOy8sHA1fm5Y/k8isCH8zHWa4N4t258HP6pVK8Pf1stDjesaQZD8r3XRN4On9fIy+v0ep4y8p/FbioVfc3n/NjpBRvs7vZvidwCym36UeB+1p1f91yqs6+wCV5+RJgvwrlDwBuiYg3GhpVz2qN+R2SRMokX0p8W9P+vVQx3oiYExFP5OXngZeAZs5dMgZ4MiKejoi/A78mxV1UvI6rgV3z/dwX+HVELIyIucCT+XgtjTciflf4Ob0XWLfBMfWkmvvbnT3IsxVExF+A24FPNijOklrjPQS4osEx9SgiJpH+cO7OvsClkdwLDJa0Ni24v66cqvO+iHgBIH9/b4XyB7PsD+EZuZl8lqQVGxFkmWpjXklpAsV7S92QwFrAKxHxj/z5OeADjQ23tnssaQzpr9WnCqsbfY8/QJrvq6Sr+/JOmXz/XiXdz2r2rbdaz/kF0l/NJV39bDRStfF+Jv87Xy1pvRr3raeqz5m7Sz8I3FlY3ez7W43urqnp93dAvYTbE0m/Bd7fxaZv1nictYGRpKzqJd8A/kT6ZXo+cBLw7d5FutS56hHz+pGmCdkQuFNp/q2/dlGuz+8c1PkeX0aacHJxXt2Qe1x+6i7Wld+X7spUs2+9VX1OSZ8lzdP28cLqZX42IuKprvavk2rivRG4IiIWSjqG1Erdpcp9662Wcx4MXB0RbxfWNfv+VqNtfn5dOWURsVt32yS9KGntiHgh/2J8qYdD/QtwXUQsKhz7hby4UNKvWDLbb8tjzt1jRMTTku4CtgSuITXn353/+l8XWGZm4FbEK+k9wE3AqbnboXTshtzjMs8B6xU+d3VfSmWek/RuYHVSN0o1+9ZbVeeUtBvpD4SPR0RpbrXufjYa+cuzYryRZhkouQD4fmHfncr2vavuES6tln/Tg4EvF1e04P5Wo7travr9dbdedW4gzfJL/v6/PZRdpl85/7ItPcvZD+hypEydVYxZ0hql7i9JQ0jzWP0h0hPQ37FkHq5K19yseFcAriP1iV9Vtq0Z9/gBYGOlkYwrkH7hlI+yKl7HAcCd+X7eABysNJrvg8DGwP0NiLGmeJXmSfsFsE9EvFRY3+XPRhvEu3bh4z7Ao3l5IrB7jnsN0kzaxd6LlsSbY/4QaRDB1MK6VtzfatwAfD6P2vso8Gr+w6/597fZo0U68Yv0zOAO4In8fc28fjRwYaHcMOCPwLvK9r+TNLfVbOByYFA7xAz8U45rRv7+hcL+G5J+eT4JXAWs2AbxfhZYRJrosvQ1qpn3mDSaaQ7pL9xv5nXfJv1yhzTR5lX5vt0PbFjY95t5v8eBTzXpZ7dSvL8FXizczxsq/Wy0ON7vAY/kuH5Hmj+utO+R+b4/CRzRDvHmz+OAM8v2a9X9vYI0ynURqTX0BdLkscfk7QJ+lq9nFmk285bcX6cvMjOztuNuPTMzazuunMzMrO24cjIzs7bjysnMzNqOKyczM2s7rpzMzKztuHIyM7O28/8BMgpoFWpaINQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmYHHW97/H3d/bs62RhMtkghCVkgcmigCKLBpVENsmCjztHLxFcj6j3ogefc4+PePS5HtFz8Byv95gJIewRohEFRcVMMlkhGwmJs2Sd7Mkks/b3/jE9QzPpTDphqquXz+t55klX9a+rP01IfaaquqrM3REREQHICTuAiIikDpWCiIh0UCmIiEgHlYKIiHRQKYiISAeVgoiIdFApiIhIB5WCiIh0CLQUzGymmW01s+1m9kCc50ea2ctmttbMNpjZB4PMIyIiXbOgzmg2s1zgDeAmoBZYBcx1900xYx4F1rr7z8zsMmCZu4/uarmDBw/20aO7HCIiIp2sXr36gLsXn21cXoAZpgHb3X0HgJktBmYDm2LGONA3+rgfsPtsCx09ejSVlZXdHFVEJLOZWVUi44LcfVQC1MRM10bnxfoOcLeZ1QLLgC/EW5CZ3WNmlWZWWVdXF0RWEREh2FKwOPM676uaC/zS3UcAHwR+ZWanZXL3R929zN3LiovPuvUjIiLnKchSqAVKY6ZHcPruoU8DSwDc/W9AETA4wEwiItKFIEthFTDOzMaYWQEwB1jaaUw1cAOAmV1KWylo/5CISEgCKwV3bwEWAMuBzcASd99oZg+Z2azosK8AnzWz9cBjwCdcN3gQEQlNkN8+wt2X0XYAOXbegzGPNwFXB5lBREQSpzOaRUSkg0pBRCTFuTv//MImNu0+Fvh7qRRERFLcq28e5Od/3snWfSoFEZGsV15RxYCe+dw8YXjg76VSEBFJYfuPNfC7jfu4s6yUovzcwN9PpSAiksIeX1VDS8SZN21kUt5PpSAikqJaI85jK6u5dtxgRg/ulZT3VCmIiKSol7fsZ/fRBuZPH5W091QpiIikqIUVVQztW8iNlw5J2nuqFEREUlDNoZP86Y065kwdSV5u8lbVKgURkRS0aGU1OWbMmVZ69sHdSKUgIpJimloiLFlVww2XDGF4vx5JfW+VgohIivntxr0crG9i/ozkHWBup1IQEUkx5SuqGDmwJ9delPx7jqkURERSyLZ9x6nYeYh500eSkxPvrsbBUimIiKSQ8opqCnJzuPOqEaG8v0pBRCRFnGxq4ak1tdx8xTAG9S4MJYNKQUQkRTy/fg/HG1q4O4QDzO1UCiIiKWJhRRUXD+1N2agBoWVQKYiIpIANtUfYUHuUu2eMwiz5B5jbqRRERFJA+YpqeuTn8pEpJaHmUCmIiITs6Klmlq7fzUemXEDfovxQs6gURERC9syaWk41tyb1EtlnEmgpmNlMM9tqZtvN7IE4z//IzNZFf94wsyNB5hERSTXuTnlFNZNK+zOhpF/YccgLasFmlgs8AtwE1AKrzGypu29qH+PuX4oZ/wVgSlB5RERS0cqdh9i2/wTfv2Ni2FGAYLcUpgHb3X2HuzcBi4HZXYyfCzwWYB4RkZRTXlFN36I8bpl4QdhRgGBLoQSoiZmujc47jZmNAsYALwWYR0QkpRw40chvXt/D7VeNoEdBbthxgGBLId4Xbf0MY+cAT7p7a9wFmd1jZpVmVllXV9dtAUVEwvREZS3NrZ4SB5jbBVkKtUDsLYNGALvPMHYOXew6cvdH3b3M3cuKi4u7MaKISDgiEWfRyipmjB3IRUN6hx2nQ5ClsAoYZ2ZjzKyAthX/0s6DzGw8MAD4W4BZRERSyivb6qg5dCrU6xzFE1gpuHsLsABYDmwGlrj7RjN7yMxmxQydCyx29zPtWhIRyTgLV1QzuHch779sWNhR3iawr6QCuPsyYFmneQ92mv5OkBlERFLN7iOneGnLPj5/3YUU5KXWOcSplUZEJAssXlmNA3Omjgw7ymlUCiIiSdTcGmHxqhreN34IpQN7hh3nNCoFEZEk+v2mfew/3sj86am3lQAqBRGRpCqvqKakfw+uGz8k7ChxqRRERJJk54F6/rL9AHOnlZKbE96NdLqiUhARSZJFFVXk5RgfnVp69sEhUSmIiCRBQ3MrT6yu5QOXD2NIn6Kw45yRSkFEJAle2LCHIyebmT8jNQ8wt1MpiIgkQXlFFWOLe/GusYPCjtIllYKISMA27T7GmuojzJ8+CrPUPMDcTqUgIhKw8ooqCvNyuP3KuLeUSSkqBRGRAJ1obOHZtbu4ZdIF9O9ZEHacs1IpiIgE6Nm1u6hvak3ZM5g7UymIiATE3Vm4oorLL+jL5NL+YcdJiEpBRCQga6qPsGXv8bQ4wNxOpSAiEpDyFVX0Lsxj9uQLwo6SMJWCiEgADtc38fxre7h1Sgm9CgO9n1m3UimIiATgydW1NLVEUv4M5s5UCiIi3SwScRatrKZs1AAuGdY37DjnRKUgItLNXn3zIDsP1HP3jFFhRzlnKgURkW5WXlHFgJ75zJwwLOwo50ylICLSjfYda+B3m/bx0bJSivJzw45zzlQKIiLd6PFVNbRGnLnT0usAczuVgohIN2lpjfDYymquHTeY0YN7hR3nvARaCmY208y2mtl2M3vgDGM+amabzGyjmS0KMo+ISJBe3lrHnqMNzJ+efgeY2wV2RoWZ5QKPADcBtcAqM1vq7ptixowDvgFc7e6HzWxIUHlERIK2cEUVQ/sWcuOl6bsqC3JLYRqw3d13uHsTsBiY3WnMZ4FH3P0wgLvvDzCPiEhgqg+e5JVtdcyZOpK83PTdMx9k8hKgJma6Njov1sXAxWb2VzNbYWYz4y3IzO4xs0ozq6yrqwsorojI+Vu0spocs7Q9wNwuyFKId0lA7zSdB4wDrgPmAv9pZqddX9bdH3X3MncvKy4u7vagIiLvRGNLK09U1nDDJUMY1q8o7DjvSJClUAuUxkyPAHbHGfOcuze7+05gK20lISKSNn77+l4O1jel5RnMnQVZCquAcWY2xswKgDnA0k5jngXeB2Bmg2nbnbQjwEwiIt2uvKKaUYN6cs1Fg8OO8o4FVgru3gIsAJYDm4El7r7RzB4ys1nRYcuBg2a2CXgZ+Jq7Hwwqk4hId3tj33FW7jzEvGkjyclJjxvpdCXQi3y7+zJgWad5D8Y8duDL0R8RkbSzqKKagtwc7rhqRNhRukX6fm9KRCRkJ5taeGp1LR+8YhiDeheGHadbqBRERM7Tr9fv5nhjC/Mz4ABzO5WCiMh5WriimvFD+1A2akDYUbqNSkFE5DxsqD3Ca7uOMn/GSMzS/wBzO5WCiMh5WLiiip4Fudw6pfOFGtKbSkFE5BwdPdXM0vW7mT25hD5F+WHH6VYqBRGRc/T0mloamiPMn57e1zmKR6UgInIO3J3yimoml/ZnQkm/sON0O5WCiMg5qNh5iO37T2TkVgKoFEREzkl5RTV9i/K4ZdIFYUcJhEpBRCRBdccb+e3re7jjqlKK8nPDjhMIlYKISIKeWF1Dc6szf0Zm7joClYKISEJaI86iimreNXYQFxb3DjtOYFQKIiIJeGVbHbWHT2XEjXS6olIQEUlA+YoqBvcu5KbLhoYdJVAqBRGRs9h15BQvbdnPnKmlFORl9mozsz+diEg3WLyyGgfmTCs969h0p1IQEelCc2uExatquH78EEYM6Bl2nMCpFEREuvDipn3UHW/M6K+hxlIpiIh0obyiipL+PXjvxUPCjpIUKgURkTPYUXeCv24/yLzpI8nNyZwb6XRFpSAicgaLKqrJyzE+Wpb5B5jbqRREROJoaG7lidW1fGDCMIr7FIYdJ2kCLQUzm2lmW81su5k9EOf5T5hZnZmti/58Jsg8IiKJemHDHo6eaubu6Zl9BnNneUEt2MxygUeAm4BaYJWZLXX3TZ2GPu7uC4LKISJyPhZWVHFhcS9mjB0YdpSkCnJLYRqw3d13uHsTsBiYHeD7iYh0i427j7K2+gjzp4/CLDsOMLcLshRKgJqY6drovM5uN7MNZvakmWXP0RwRSVnlFdUU5edw+5Ujwo6SdEGWQrx69U7TvwZGu/tE4PfA/4u7ILN7zKzSzCrr6uq6OaaIyFtONLbw3Npd3DLxAvr1zA87TtIlXApmdo2ZfTL6uNjMxpzlJbVA7G/+I4DdsQPc/aC7N0Ynfw5cFW9B7v6ou5e5e1lxcXGikUVEztkza3dR39TK/Ay/RPaZJFQKZvZt4OvAN6Kz8oGFZ3nZKmCcmY0xswJgDrC003KHx0zOAjYnkkdEJAjuTvmKKiaU9GXSiH5hxwlFolsKt9K20q4HcPfdQJ+uXuDuLcACYDltK/sl7r7RzB4ys1nRYfeZ2UYzWw/cB3zi3D+CiEj3WFN9mC17j2flAeZ2iX4ltcnd3cwcwMx6JfIid18GLOs078GYx9/gra0PEZFQla+opk9hHrMmXRB2lNAkuqWwxMz+A+hvZp+l7aDwz4OLJSKSXIfrm3j+tT3cemUJvQoDO4Ur5SX0yd39B2Z2E3AMGA886O4vBppMRCSJnlxdS1NLhPlZdgZzZwmVQnR30Uvu/qKZjQfGm1m+uzcHG09EJHiRiFNeUcXU0QMYP6zLw6UZL9HdR68AhWZWQtuuo08CvwwqlIhIMr365kH+fvAkd2fp11BjJVoK5u4ngduAf3P3W4HLgoslIpI8C1dUMbBXATMnDAs7SugSLgUzexcwH3ghOi97j8SISMbYd6yBFzfv486yERTm5YYdJ3SJlsL9wAPA09FzDcYALwUXS0QkORavrKE14syblh33YD6bRH/bPwlEgLlmdjdt1zXqfB0jEZG00tIaYfGqat5zcTGjBiV0+lXGS7QUyoGvAq/TVg4iImnvpS372XO0ge/MujzsKCkj0VKoc/dfB5pERCTJyiuqGda3iBsuGRJ2lJSRaCl828z+E/gD0H5VU9z96UBSiYgErPrgSV7ZVsf9N4wjL1e3q2+XaCl8EriEtqujtu8+ckClICJpadHKanLMmDNVB5hjJVoKk9z9ikCTiIgkSWNLK0sqa7jx0iEM61cUdpyUkug20woz08lqIpIRfvv6Xg7VN+kM5jgS3VK4Bvi4me2k7ZiCAR69jaaISFopX1HNqEE9ufrCwWFHSTmJlsLMQFOIiCTJG/uOs/Lvh/jmBy8hJyc7b6TTlUQvnV0VdBARkWQoX1FFQV4Od1xVevbBWUjfwxKRrFHf2MLTa3bxoSuGM7BXQdhxUpJKQUSyxq/X7+Z4Ywvzp+trqGeiUhCRrODuLKyo4pJhfbhq1ICw46QslYKIZIUNtUd5fdcx5k8fiZkOMJ+JSkFEssLCFVX0LMjlI1NKwo6S0lQKIpLxjp5s5tcbdjN7cgl9ivLDjpPSVAoikvGeWlNLQ3NEB5gTEGgpmNlMM9tqZtvN7IEuxt1hZm5mZUHmEZHs4+6UV1QxubQ/E0r6hR0n5QVWCmaWCzwC3AxcRttd2067fpKZ9QHuAyqCyiIi2WvFjkO8WVev6xwlKMgthWnAdnff4e5NwGJgdpxx3wW+DzQEmEVEslR5RRX9euTz4YnDw46SFoIshRKgJma6Njqvg5lNAUrd/fmuFmRm95hZpZlV1tXVdX9SEclIdccbWb5xL3dcNYKi/Nyw46SFIEsh3heBveNJsxzgR8BXzrYgd3/U3cvcvay4uLgbI4pIJltSWUNzqzNPB5gTFmQp1AKxV5waAeyOme4DTAD+aGZ/B2YAS3WwWUS6Q2vEWVRRzbsvHMSFxb3DjpM2giyFVcA4MxtjZgXAHGBp+5PuftTdB7v7aHcfDawAZrl7ZYCZRCRLvPJGHbuOnGL+dB1gPheBlYK7twALgOXAZmCJu280s4fMbFZQ7ysiAm1nMBf3KeT9lw8NO0paSfQmO+fF3ZcByzrNe/AMY68LMouIZI/awyd5aet+7r3uIvJzdY7uudB/LRHJOItX1mDAXB1gPmcqBRHJKM2tERavquF944dQ0r9H2HHSjkpBRDLK7zbu48CJRp3BfJ5UCiKSUcorqijp34P3XKxzms6HSkFEMsabdSd49c2DzJs+ktwc3UjnfKgURCRjLKqoJj/X+GhZ6dkHS1wqBRHJCA3NrTy5upYPXD6M4j6FYcdJWyoFEckIz2/Yw9FTzTqD+R1SKYhIRli4oooLi3sxY+zAsKOkNZWCiKS913cdZV3NEeZPH4WZDjC/EyoFEUl75RXVFOXncPuVI8KOkvZUCiKS1o43NPPcul3cMvEC+vXMDztO2lMpiEhae3btLk42teoM5m6iUhCRtOXulFdUM6GkLxNH9As7TkZQKYhI2lpddZgte49ztw4wdxuVgoikrfKKavoU5jFr8gVhR8kYKgURSUuH6pt4YcMebruyhJ4Fgd4vLKuoFEQkLT25uoam1gjzdYC5W6kURCTtRCJtB5injR7IxUP7hB0no6gURCTt/PXNA1QdPMn8GbrdZndTKYhI2vnV36oY1KuAmROGhR0l46gURCRtnGxq4StL1vO7TfuYN30khXm5YUfKODpkLyJpYcveY9xbvoYdB+q574Zx3H/DuLAjZaRAtxTMbKaZbTWz7Wb2QJznP2dmr5nZOjP7i5ldFmQeEUk/7s7ildXM/slfOdbQQvmnp/Plmy7W7TYDEtiWgpnlAo8ANwG1wCozW+rum2KGLXL3f4+OnwX8EJgZVCYRSS8nGlv41jOv8dy63Vxz0WB+dNdk3VUtYEHuPpoGbHf3HQBmthiYDXSUgrsfixnfC/AA84hIGtm4+ygLFq2l6mA9X33/xXz+uou0dZAEQZZCCVATM10LTO88yMzuBb4MFADXB5hHRNKAu7NwRRXffWEzA3rm89hnZzB97KCwY2WNIEshXqWftiXg7o8Aj5jZPOB/Ah8/bUFm9wD3AIwcqe8li2SqYw3NPPDUBpa9tpfrxhfzr3dOYlBv7S5KpiBLoRYojZkeAezuYvxi4GfxnnD3R4FHAcrKyrSLSSQDra85woLH1rD7SAPfuPkSPnvtWHK0uyjpgiyFVcA4MxsD7ALmAPNiB5jZOHffFp38ELANEckq7s4v/vp3vvebzQzpU8SSf3gXV40aEHasrBVYKbh7i5ktAJYDucAv3H2jmT0EVLr7UmCBmd0INAOHibPrSEQy15GTTXztyQ28uGkfN146lB/cOZH+PQvCjpXVAj15zd2XAcs6zXsw5vH9Qb6/iKSuNdWH+cKitew/3sD/+vBlfOrq0bpRTgrQGc0iklSRiPPzP+/g4eVbGd6/iCc/924mlfYPO5ZEqRREJGkO1TfxlSXreHlrHTdPGMb3bp9Ivx75YceSGCoFEUmKlTsPcd9jazlU38R3Z1/O3TN0X+VUpFIQkUBFIs5P/7idH774BiMH9uTp//FuJpT0CzuWnIFKQUQCU3e8kS8vWceftx1g1qQL+N+3XUHvQq12Upn+dkQkEK9uP8D9j6/j2Klm/uW2K5gztVS7i9KASkFEulVrxPnxH7bx45e2MXZwL3716WlcMqxv2LEkQSoFEek2+441cP/itazYcYjbrizhu7Mn0Eu7i9KK/rZEpFu88kYdX3p8HSebWvnBnZO446oRYUeS86BSEJF3pKU1wg9ffIOf/vFNxg/tw0/mTWHc0D5hx5LzpFIQkfO2+8gp7ntsLZVVh5kztZRv33I5PQpyw44l74BKQUTOy0tb9vHlJetpbonwf+ZMZvbkkrAjSTdQKYjIOWlqifDw8i38/M87uXR4Xx6ZN4Wxxb3DjiXdRKUgIgmrOXSSLzy2lnU1R/jYjFF860OXUpSv3UWZRKUgIglZvnEvX3tiPe7wyLwr+dDE4WFHkgCoFESkS40trfzLsi388tW/M3FEP34y90pGDuoZdiwJiEpBRM6o6mA9Cxat5bVdR/nU1WP4+s3jKczT7qJMplIQkbie37CbB556jRyDRz92Fe+/fFjYkSQJVAoi8jYNza189/lNlFdUM2Vkf/5t7hRGDNDuomyhUhCRDm/WneDe8jVs2Xucf3jPWL76gfHk5+aEHUuSSKUgIgA8u3YX33zmNQrzcvjFJ8q4/pKhYUeSEKgURLLcqaZWvrN0I49X1jB19AB+PHcKw/v1CDuWhESlIJLFtu07zr2L1rBt/wkWvO8ivnjjOPK0uyirqRREspC788TqWh587nV6F+bx35+axrXjisOOJSkg0F8JzGymmW01s+1m9kCc579sZpvMbIOZ/cHMRgWZR0SgvrGFryxZzz8+uYEppQNYdt+1KgTpENiWgpnlAo8ANwG1wCozW+rum2KGrQXK3P2kmX0e+D5wV1CZRLLd5j3HuHfRGnYeqOeLN47jC9ePIzdH902WtwS5+2gasN3ddwCY2WJgNtBRCu7+csz4FcDdAeYRyVruzqKV1fzTrzfRr0c+5Z+ZzrsvHBx2LElBQZZCCVATM10LTO9i/KeB38R7wszuAe4BGDlyZHflE8kKxxua+cbTr/H8hj1cO24wP7prMoN7F4YdS1JUkKUQb5vU4w40uxsoA94b73l3fxR4FKCsrCzuMkTkdK/vOsqCRWuoOXyKr31gPJ9/74XkaHeRdCHIUqgFSmOmRwC7Ow8ysxuBbwHvdffGAPOIZA1357//VsU/v7CZgb0KWHzPDKaOHhh2LEkDQZbCKmCcmY0BdgFzgHmxA8xsCvAfwEx33x9gFpGscfRUM19/cgO/3biX6y8Zwg/unMTAXgVhx5I0EVgpuHuLmS0AlgO5wC/cfaOZPQRUuvtS4GGgN/CEmQFUu/usoDKJZLp1NUdYsGgNe4828M0PXsJnrhmr3UVyTgI9ec3dlwHLOs17MObxjUG+v0i2cHf+6y87+d5vtjC0bxFLPvcurhw5IOxYkoZ0RrNIGotEnAMnGvnmM6/x+837ef9lQ3n4jkn065kfdjRJUyoFkSRzd041t3KisYUTDS1tf8Y8rm9s4Xj0zxMNMY87ja9vbFsGQH6u8e1bLuMT7x5NdFesyHlRKYgkwN1pbIm8tdJuiFlRx1lhn2hoob6pbdyJOCv4SAJfrM7LMXoX5dGrII8+RXn0Ksyjf88CRgzsSe+CPHoX5dG7sO3nmnGDuXR43+D/Q0jGy5pSeGp1Lf/31Z3kmmFm5BjkmJFjhrU/ziE63fb8aWNziE6/Na/9tblvez7OsjuWEfva2LGnLzt2bLznO+fOiXk+0dxxx8dd9unLADp9ngSXGTM+6N9qm1sjb19Zx660o49jV/Cxv6HHjq9vbKG59exrcjM6VtS9C9tW5H2K8hjWt4he0XntK/jYcbEr+PbHhXk5+q1fki5rSqFnQS5D+hQRcSfibb/5RdyJRCDiTmvEaW5tm9ca53l3oq+NfQytEY+O5fRl+9tf2zb2reUk8ttiprPTSq+rEolTkjGFaNHltUS8Y6Xe2BJJKEevgty2FXXMynlkr55vW0m3r+A7Hkf/7F301uOeBblakUtay5pSuPmK4dx8xfCwY5zm7YXy9gJ5q2DiF1Ik0rl8YsZG4r+my+W1LyOS+Hjivf5sBRk5x/Ex8+Lncxw6XpOXY2+tzAtO/y28fYXe/rhXQZ4uCicSlTWlkKrMjFyD3LhXBRERSS7dYklERDqoFEREpINKQUREOqgURESkg0pBREQ6qBRERKSDSkFERDqoFEREpIO5p9e1FsysDqg6z5cPBg50Y5x0oM+cHfSZs8M7+cyj3L34bIPSrhTeCTOrdPeysHMkkz5zdtBnzg7J+MzafSQiIh1UCiIi0iHbSuHRsAOEQJ85O+gzZ4fAP3NWHVMQEZGuZduWgoiIdCHrSsHMHjazLWa2wcyeMbP+YWcKmpndaWYbzSxiZhn9bQ0zm2lmW81su5k9EHaeoJnZL8xsv5m9HnaWZDCzUjN72cw2R/+fvj/sTEEzsyIzW2lm66Of+Z+CfL+sKwXgRWCCu08E3gC+EXKeZHgduA14JewgQTKzXOAR4GbgMmCumV0WbqrA/RKYGXaIJGoBvuLulwIzgHuz4O+4Ebje3ScBk4GZZjYjqDfLulJw99+5e0t0cgUwIsw8yeDum919a9g5kmAasN3dd7h7E7AYmB1ypkC5+yvAobBzJIu773H3NdHHx4HNQEm4qYLlbU5EJ/OjP4EdDM66UujkU8Bvwg4h3aYEqImZriXDVxjZzMxGA1OAinCTBM/Mcs1sHbAfeNHdA/vMGXmPZjP7PTAszlPfcvfnomO+RdumaHkyswUlkc+cBeLd6Fpfr8tAZtYbeAr4orsfCztP0Ny9FZgcPQb6jJlNcPdAjiNlZCm4+41dPW9mHwc+DNzgGfKd3LN95ixRC5TGTI8AdoeURQJiZvm0FUK5uz8ddp5kcvcjZvZH2o4jBVIKWbf7yMxmAl8HZrn7ybDzSLdaBYwzszFmVgDMAZaGnEm6kZkZ8F/AZnf/Ydh5ksHMitu/JWlmPYAbgS1BvV/WlQLwE6AP8KKZrTOzfw87UNDM7FYzqwXeBbxgZsvDzhSE6BcIFgDLaTsAucTdN4abKlhm9hjwN2C8mdWa2afDzhSwq4GPAddH//2uM7MPhh0qYMOBl81sA22/+Lzo7s8H9WY6o1lERDpk45aCiIicgUpBREQ6qBRERKSDSkFERDqoFEREpINKQbKGmZ04+6guX/+kmY09y5g/nu1KtImM6TS+2Mx+m+h4kXdCpSCSADO7HMh19x3Jfm93rwP2mNnVyX5vyT4qBck61uZhM3vdzF4zs7ui83PM7KfRa9Y/b2bLzOyO6MvmA8/FLONnZlbZ1fXtzeyEmf2rma0xsz+YWXHM03dGr5H/hpldGx0/2sz+HB2/xszeHTP+2WgGkUCpFCQb3Ubbdekn0XbJgIfNbHh0/mjgCuAztJ0B3u5qYHXM9LfcvQyYCLzXzCbGeZ9ewBp3vxL4E/DtmOfy3H0a8MWY+fuBm6Lj7wJ+HDO+Erj23D+qyLnJyAviiZzFNcBj0StP7jOzPwFTo/OfcPcIsNfMXo55zXCgLmb6o2Z2D23/hobTdlOfDZ3eJwI8Hn28EIi9eFv749W0FRG0XSf/J2Y2GWgFLo4Zvx+44Bw/p8g5UylINop3ie2u5gOcAooAzGwM8FXP2Bb8AAABRUlEQVRgqrsfNrNftj93FrHXlGmM/tnKW/8OvwTso20LJgdoiBlfFM0gEijtPpJs9ApwV/TGJcXAe4CVwF+A26PHFoYC18W8ZjNwUfRxX6AeOBodd/MZ3icHaD8mMS+6/K70A/ZEt1Q+BuTGPHcxAV0qWSSWthQkGz1D2/GC9bT99v6P7r7XzJ4CbqBt5fsGbXf0Ohp9zQu0lcTv3X29ma0FNgI7gL+e4X3qgcvNbHV0OXedJddPgafM7E7g5ejr270vmkEkULpKqkgMM+vt7ifMbBBtWw9XRwujB20r6qujxyISWdYJd+/dTbleAWa7++HuWJ7ImWhLQeTtno/e0KQA+K677wVw91Nm9m3a7vlcncxA0V1cP1QhSDJoS0FERDroQLOIiHRQKYiISAeVgoiIdFApiIhIB5WCiIh0UCmIiEiH/w8yfTig7zreHwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#绘制重要系数\n",
    "coefs=pd.Series(ridge.coef_,index=X1_train.columns)\n",
    "\n",
    "#正系数值最大的10个特征和负系数最小值(绝对值大)的10个特征\n",
    "imp_coefs=pd.concat([coefs.sort_values().head(10),coefs.sort_values().tail(10)])\n",
    "imp_coefs.plot(kind=\"barh\")\n",
    "plt.title(\"Coefficients in the Ridge Model\")\n",
    "plt.show()\n",
    "\n",
    "mse_mean=np.mean(ridge.cv_values_,axis=0)\n",
    "plt.plot(np.log10(alphas),mse_mean.reshape(len(alphas),1))\n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评价：训练集要好点，测试集稍差"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.3Lasso模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha: 1.0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaoAAAEICAYAAAAOW7ATAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XuYHFWd//H3R+4QJEKiglwiBIxCIJgBZAFNIIs3BFzDw22VIOqisvxc5LbAYhRRXFREFLmJ3LIsgqBB0IBkQ0IIlwRyFQQkYZMFIUGuChHI9/fHOW0qTc9M90xPd0/P5/U8eeiuOlV1qjLMyTl16lOKCMzMzFrVW5pdATMzs664oTIzs5bmhsrMzFqaGyozM2tpbqjMzKyluaEyM7OW5obK+jVJ75H0oKSXJB0vaQNJN0t6QdL1ko6UdFsV+zlN0mWNqHMXddha0suS1qrT/iZKuqYe+xpoJC2RNK6KcsMkhaS1G1GvgcoNlTWEpCMkzc6/iJ+S9BtJe9dh1ycD0yJi44j4ITAeeAewWUQcEhGTImL/7nYSEd+KiM/1tjK9+cUVEf8bEYMi4o0eHHeMpGW1blfD/q+Q9M2+2n9P5XqFpAPLlv8gL5/QpKpZHbmhsj4n6QTgB8C3SI3I1sCFwEF12P02wKKy749ExOt12Lf1D48AR5W+5H8kHAL8sWk1srpyQ2V9StImwDeAL0fEjRHxl4h4LSJujoiTcpn18r+An8x/fiBpvcI+DpA0V9Lzku6WtHNePhUYC/wo99SuBc4EDs3fj5E0QdJdhX3tKOl2SX+W9LSk0/LyNYbJJH0gH+t5SfMkjSmsmybpLEkz85DjbZKG5NXT83+fz3XYU9JwSXfm4cgVkq7r5Fqt0Rvr5jjF7TYCfgNskY/5sqQt8up1JV2Vt18kqaOw3RaSfiFpuaTFko6v5u+0wvHPl7RU0ouS5kjap7Bu99yTfjFf7+/n5etLukbSs/ka3y/pHYV6Tc5/R49J+nw3VbgZ2EvS2/L3jwDzgT8V6vEWSWdIekLSM/mabFJY/+m87llJp5ed31sknSrpj3n9zyVt2pNrZT3jhsr62p7A+sBNXZQ5HfgAMArYBdgdOANA0vuBy4F/ATYDLgYmS1ovIvYFZgDH5SGzw0m9tuvy958WDyJpY+B3wG+BLYDhwB3llZH0LuAW4JvApsCJwC8kDS0UOwI4Gng7sG4uA/DB/N/BuQ6zgLOA24C3AVsCF3RxLcp1dpy/i4i/AB8FnszHHBQRT+bVBwL/DQwGJgM/yuf4FtIv+HnAu4D9gK9I+nANdSu5n/R3tynwX8D1ktbP684Hzo+ItwLbAT/Py48CNgG2Iv29Hgu8ktddCywj/R2NB74lab8ujv9qPrfD8vfPAFeVlZmQ/4wFtgUGsfpavA/4CfDpfMzNSH9PJccDBwMfyuufA37cRX2sztxQWV/bDFjRzVDckcA3IuKZiFgOfJ30SwPg88DFEXFvRLwREVcCK0kNW60OAP4UEd+LiFcj4qWIuLdCuX8Gbo2IWyNiVUTcDswGPlYo87OIeCQiXiH98h3VxXFfIw1JbpGPe1cXZcvVcpxK7srn8QZwNekfAgC7AUMj4hsR8beIeBy4lNW/7KsWEddExLMR8XpEfA9YD3hPXv0aMFzSkIh4OSLuKSzfDBie/17nRMSLkrYC9gZOyddqLnAZq38eOnMV8JncS/oQ8Muy9UcC34+IxyPiZeDfgcNy73U88OuImB4RK4H/AFYVtv0X4PSIWJbXTwTGyxMoGsYNlfW1Z4Eh3fxPvQXwROH7E3kZpF/wX83DQ89Lep70r/AtqN1WVHffYhvgkLJj7g1sXijzp8Lnv5L+hd6ZkwEB9+Xht8/WUOdajlPN9uvnv4ttSEOFxXM8jXQPsSaSvirpoTy0+Typp1QaojwG2AF4OA/vHZCXXw1MAf5babj3PyWtQ/p7/XNEvFQ4xBOkXl+ncuM/lNQT/3Vu2Isq/Yytnc93C2BpYV9/If3clmwD3FS4Tg8Bb9CDa2U9438RWF+bRRqaORi4oZMyT7LmpIit8zJIv0DOjoiz61CXpcDhVZa7OiK6uzdSyZteRxARfyL1DFGa6fg7SdMj4rEe7L/q43ZjKbA4IrbvzUHz/ahTSEOHiyJilaTnSA0zEfEocHgeavwn4AZJm+XG4OvA1yUNA24F/kAaIt1U0saFxmpr4P+qqM41pHuUYyusK/2MlWwNvA48DTwFvLdwThuSenslS4HPRsTMCuc/rIp6WS+5R2V9KiJeIP3y+LGkgyVtKGkdSR+V9J+52LXAGZKG5skCZ5J+6UAajjpW0h5KNpL08Xy/qVa/Bt4p6StKEzg2lrRHhXLXAJ+Q9GFJa+Ub/2MkbVmhbLnlpGGjbUsLJB1S2PY5UqNS8xT0bjwNbFacINCN+4AXJZ2i9OzZWpJ2krRbF9uUrkXpz7rAxqRf+MuBtSWdCby1tIGkf5Y0NCJWAc/nxW9IGitppNIzYy+ShgLfiIilwN3At/Mxdib1yiZVcU4/BP6R1RNaiq4F/k3SuyUNYvW9zNdJ/4A6QNLe+Zy+wZq/Gy8Czpa0TT6noZLqMWPVquSGyvpcRHwfOIE0LLOc9C/U41h9H+GbpHtA84EFwAN5GRExm9Qb+RHpl/xjpJviPanHS6RfZJ8gDYk9SoV/fedflgeRhsJK9T2JKv5/iYi/AmcDM/NQ0QdI94PulfQy6ab//4uIxT05hy6O+zDpl/Hj+bhdDo3me1afIN3zWgysIN0L6qqhO5U04aH0Zypp+O43pCniT5B6z0sL23wEWJTP/XzgsIh4FXgnqYF4kTSUdier/3FyODCM1Au6Cfhavk/Y3TX4c0TcEZVfsnc5abhxej7fV4F/zdstAr5MmgjyFOnnrPhM2vmkv7fbJL0E3ANU+geO9RH5xYlmZtbK3KMyM7OW5obKzMxamhsqMzNraW6ozMyspfk5qh4aMmRIDBs2rNnVMDPrV+bMmbMiIoZ2X3I1N1Q9NGzYMGbPnt3sapiZ9SuSnui+1Jo89GdmZi2t3/aoJA0GjoiIC5tdFxvY7pi6XbOrYNZQ++3b2Fd99ece1WDgS82uhJmZ9a3+3FCdA2yn9EK9cyWdlNOZ50v6Ovz9RXQPS7pM0kJJkySNU3oR3aOSds/lJkq6WtLUvLwnYaRmZtYH+nNDdSrwx4gYBdwObE964d4oYLSk0gvshpOyunYGRpBeRLc36QV0pxX2tzPwcdKL/s6slJUm6QtKbyudvXz58r45KzMzW0N/bqiK9s9/HiQFmo4gNVyQXmWwIKc3LwJKoZULSMGXJb+KiFciYgXwP6RGbw0RcUlEdEREx9ChNc2uNDOzHuq3kynKCPh2RFy8xsL0rpiVhUWrCt9Xseb5l6fzOq3XzKwF9OeG6iXSu3AgvWrgLEmTIuJlSe8ivd+mFgdJ+jawETCGNLRo1q1Gz4AyG2j6bUMVEc/mSRELSe/D+S9gliSAl4F/praX090H3EJ68+dZEfFkN+XNzKwB+m1DBRARR5QtOr9CsZ0K5ScUPi8prgMeiYgv1LN+ZmbWe+0ymcLMzNpUv+5R1UtETGx2HczMrDI3VGa9NHHixGZXwfoh/9xUz0N/nZC0VrPrYGZmA7hHJeksYEVEnJ+/nw08DXwSeIqUcPG+5tXQzMxgYPeofgocBSDpLcBhwP+REilOj4g3NVKOUDIza7wB21Dl6enPStqV1fFLzwL3RcTiTrZxhJKZWYMN2KG/7DJgAvBO4PK87C9Nq431S74pbta3BmyPKrsJ+AiwGymGyczMWsyA7lFFxN8k/Q/wfES8keOXzMyshQzohipPovgAcAhAREwDpjWxSmZmVmbADv1Jeh/wGOn9VI82uz5mZlZZv22oJJ3WfanORcTvI2LbiPhqvepkZmb115+H/k4DvtXsSpgtO3VGs6vQUFues0+zq2ADTL/oUUn6paQ5khblh27PATaQNFfSpFzmnyXdl5ddXIpAkvSypO/k7X8naXdJ0yQ9LunAXGaCpF9J+q2kP0j6WhNP18zMCvpFQwV8NiJGAx3A8cC5wCsRMSoijpT0XuBQYK+IGEV6YeKReduNgGl5+5eAbwL/SIpK+kbhGLvnbUYBh0jqaMB5mZlZN/rL0N/xkj6ZP28FbF+2fj9gNHB/nmK+AfBMXvc34Lf58wJgZUS8JmkBMKywj9sj4lkASTcCewOziweR9AXgCwBbb71178/KzMy61fINlaQxwDhgz4j4q6RpwPrlxYArI+LfK+zitYiI/HkVsBIgIlZJKp5/lG1X/p2IuAS4BKCjo+NN683MrP5avqECNgGey43UCNJzTwCvSVonIl4D7gB+Jem8iHhG0qbAxhHxRA3H+ce83SvAwcBn63kS1r48ucCsb/WHe1S/BdaWNB84C7gnL78EmC9pUkT8HjgDuC2Xux3YvMbj3AVcDcwFfhERs7spb2ZmDaDVo2IDl6QJQEdEHFftNh0dHTF7ttsyM7NaSJoTETVNVusPPSozMxvA+sM9qj4XEVcAVzS5GmZmVoF7VGZm1tLcoyojaTKwbUTs1Oy6WP/wvUMPaHYVuvXV637d7CqY9Zh7VAWS/gl4udn1MDOz1RrWUEnaSNItkuZJWijpUEmjJd2Zc/imSNo8l/28pPtz2V9I2jAvPyRvO0/S9LxsfUk/k7RA0oOSxublEyTdmPP7HpX0n93UbxBwAiliqbMyX5A0W9Ls5cuX1+vSmJlZFxrZo/oI8GRE7JKH1X4LXACMzzl8lwNn57I3RsRuEbEL8BBwTF5+JvDhvPzAvOzLABExEjgcuFJSKbliFCkDcCRwqKStuqjfWcD3gL92ViAiLomIjojoGDp0aC3nbmZmPdTIhmoBMC4nme9DyuzbCbhd0lzSA7tb5rI7SZqR8/iOBHbMy2cCV0j6PLBWXrY36UFdIuJh4Algh7zujoh4ISJeBX4PbFOpYpJGAcMj4qb6na6ZmdVDwyZTRMQjkkYDHwO+TUqPWBQRe1YofgVwcETMyw/jjsn7OFbSHsDHgbm5gVEXh11Z+PwGnZ/vnsBoSUtymbdLmhYRY6o7OxvIPFHBrG818h7VFsBfI+Ia4LvAHsBQSXvm9etIKvWcNgaekrQOq1/XgaTtIuLeiDgTWEHqlU0vlZG0A7A18Ida6hYRP4mILSJiGKmH9ogbKTOz1tDI6ekjgXMlrQJeA74IvA78UNImuS4/ABYB/wHcSxrGW0BquMjbb0/qRd0BzAMeBi7Kw4SvAxMiYmV+3YeZmfVzzvrrIWf9mZnVzll/ZmbWdgZcMoWke4H1yhZ/OiIWNKM+ZmbWtbZpqCQNBo6IiAvz9zHAiRGxRr5NROzRyfb7kiZ5rAvMAY6JiNf7tNLWFn587NRmV6FbX75o32ZXwazH2mnobzDwpZ5sKOktwJXAYflh5CeAo+pYNzMz66GWaqgkDZP0sKTLclTSJEnjJM3MMUi7S5oo6XJJ0yQ9Lun4vPk5wHaS5ko6Ny8bJOmGvM9J6nwq4GbAyoh4JH+/HfhUhfo5QsnMrMFaqqHKhgPnAzsDI4AjSM82nQiclsuMAD4M7A58LT9vdSrwx4gYFREn5XK7Al8B3gdsC+zVyTFXAOtIKs1EGU96RmsNjlAyM2u8VmyoFkfEgohYRXqm6o5Ic+gXAMNymVsiYmVErACeAd7Ryb7ui4hleV9zC9uvIe//MOA8SfcBL5GeyTIzsyZrxckUxdijVYXvq1hd32qjkaotR0TMAvYBkLQ/q/MCzbrkiQpmfasVe1Q99RKrEyxqJunt+b/rAacAF9WpXmZm1gtt01BFxLPAzDwJ49xuN3izkyQ9BMwHbo6I1p9zbGY2ADhCqYccoWRmVjtHKJmZWdtpxckUfUrSTcC7yxafEhFTmlEf6/8eGvHeZlehovc+/FCzq2BWFw3vUUmakN9NVfq+RNKQPjjOrZIG5z9/T6yIiE/mZ62Kf6ZI+q2k5yX5LXhmZi2kGUN/E4AtuitUDUldTTf/WEQ8T/XRSucCn65HvczMrH66bagknVyKKZJ0nqSp+fN+kq6RtL+kWZIekHS9pEF5/ZmS7s+z8C5RMh7oACblqKMN8mH+NW+/QNKIvP1GOSrpfkkPSjooL5+Qj3MzcJukzSVNz/tbKKn0LFSpp1YpWulNIuIO0hT3rq6FI5TMzBqsmh7VdPKDsKRGZlCOLNqblBZxBjAuIt4PzAZOyGV/FBG75ZDXDYADIuKGXObIPOT2Si67Im//E1JUEsDpwNSI2A0YS3q770Z53Z7AURGxLyliaUpEjAJ2ISVQFFWKVuoRRyiZmTVeNZMp5gCjJW1MSnp4gNRg7QNMJuXozcx5r+sCs/J2YyWdDGwIbEqKQ7q5k2PcWDjWP+XP+wMHSio1XOsDW+fPt0fEn/Pn+4HLc+P5y4gob6jMzKwf67ahiojXJC0BjgbuJj0QOxbYDlhMajQOL24jaX3gQqAjIpZKmkhqaDpTijoqxhwJ+FRE/KFs33sAfynUb7qkDwIfB66WdG5EXNXdeZnVi2fXmfWtaidTTCcNyU0HZgDHkobY7gH2kjQcQNKGknZgdaO0It+zGl/YV7VRR1NI966U971rpUKStgGeiYhLgZ8C7y8r0qtoJTMza65qG6oZwObArIh4GngVmBERy0mz+K6VNJ/UcI3Is+0uJd3D+iVpeK7kCuCisskUlZwFrAPMl7Qwf69kDDBX0oOkd0idX1xZbbSSpBnA9cB+kpZJ+nAXdTMzswZxhFIPOULJzKx2jlAyM7O2M6AilCSNBK4uW7wyIvZoRn2sPYy8cmSvtl9w1II61cSsPfWrHlV5HJKkMTVGHn0IGER63mpcfrZqj/ww8g8lPSZpvqTyCRlmZtYk/aqhovo4pM7MBMYBT5Qt/yiwff7zBdKDx2Zm1gKaEUo7TNLDki7LM/EmSRonaaakRyXtLmlijk+aJunxUoQTleOQBkm6Ie9zUmk6eyUR8WBELKmw6iDgqkjuAQZL2rxC3R2hZGbWYM3qUQ0nTSPfGRhBikHam/Ss1mm5zAjgw8DuwNdy8kSlOKRdga+QEjK2BfbqQX3eBSwtfF+Wl63BEUpmZo3XrIZqcUQsiIhVpGilOyLNk18ADMtlbomIlRGxAngGeEcn+7ovIpblfc0tbF+LSr0wz9s3M2sBzZr1t7LweVXh+ypW16lYphit1NW+uirXlWXAVoXvWwJP9mA/NgB51p5Z3+pvkyn6Kg5pMvCZPPvvA8ALEfFUHxzHzMxq1K8aqmrjkDoj6XhJy0g9pvmSLsurbgUeBx4jRT/1ZmahmZnVkSOUesgRSmZmtXOEkpmZtZ22jFCSdBPw7rLFp0TElGbUx9rcxE16se0L9auHWZtqmx5VMV4pIj5JerZqWX7malR3jZSkKyQtzg8Tz5U0qhH1NjOzrrVNQ0Xv45UATio0bH6lvZlZC2iphqqZ8UpV1s8RSmZmDdZSDVXWzHils3N6+nmS1itf6QglM7PGa8WGqlnxSv9OagB3AzYFTun1mZiZWa+14qy/psQrFZIoVkr6GakHZ9Y9z9wz61Ot2KPqqV7FK5Ve65HvYx0MLKxTvczMrBdasUfVIxHxbJ50sRD4DXBLjbuYJGkoKUl9LnBsvetoZma1c4RSDzlCycysdo5QMjOzttM2Q3/VcryS1duwU2sdZU6WnPPxOtfErD21TUMlaTBwRERcmL+PAU6MiAOK5XK8UqXtfwp0kO5RPQJMiIiX+7TSZmbWrXYa+utthNK/RcQuEbEz8L/AcfWplpmZ9UZLNVTNjFCKiBdzHQRsALxplokjlMzMGq+lGqqsaRFK+UHfP+X9X1C+3hFKZmaN14r3qBZHxAIASX+PUJJUilCaS45QIqVIdBuhlPdVilC6q7MDR8TRktYiNVKHAj+rzylZO/OkCLO+1Yo9qqZEKJVExBvAdcCnqqmsmZn1rVZsqHqqxxFKSoaXPgOfAB6uY93MzKyHWnHor0d6GaEk4EpJb82f5wFf7INqmplZjRyh1EOOUDIzq50jlMzMrO20zdBftckUnUUoAbcB3wQOIU28+ElE/LCv6239X60RSp4laFabtmmoWJ1McWFXhbqIUDoa2AoYERGrJL29/lU0M7NatdTQXzOTKUiTJ76RX1tPRDzTh6dqZmZVaqmGKmtWMsV2wKE5Iuk3krYvL+AIJTOzxmvFhmpxRCzIPZu/J1MApWQKyMkUEbEC6DaZIu9rbmH7StYDXs2zUS4FLi8v4AglM7PGa8V7VM1KplgG/CJ/vgnHJ1mVPDnCrG+1Yo+qp3qcTJH9Etg3f/4Q6Z1UZmbWZK3Yo+qRXiZTQJqMMUnSvwEvA5+rdx3NzKx2TqboISdTmJnVzskUZmbWdtpm6K9anSVTRMSUZtTHzMy61nINlaRppOijTsfVJE0AOiLiuFr3XymZQtIHJT1AenbrsIi4odb9Wv/2zv+Z2+Nt/zR2VB1rYmblPPSX/C8wAfivJtfDzMzK9LqhknRyKcZI0nmSpubP+0m6RtL+kmZJekDS9ZIG5fWjJd0paY6kKZI2L9vvWyRdKemb+fvRkh6RdCeFhAlJn5B0r6QHJf1O0jvyto9KGlrY12OShlQ6h4hYEhHzSc9qmZlZC6lHj2o6sE/+3EHK11uHFHu0ADgDGBcR7wdmAyfk9RcA4yNiNCkF4uzCPtcGJgGPRMQZuRH7OqmB+kdSJFLJXcAHImJX4L+Bk3MSxTXAkbnMOGBeTrLoMUcomZk1Xj3uUc0BRkvamJQE8QCpwdoHmExqVGbmPNh1gVnAe4CdgNvz8rWApwr7vBj4eUSUGq89gGkRsRxA0nXADnndlsB1uTFbF1icl18O/Ar4AfBZ6pA0ERGXAJdAmp7e2/2ZmVn3et1QRcRrkpYARwN3A/OBsaSQ18XA7RFxeHEbSSOBRRGxZye7vRsYK+l7EfFq6VCdlL0A+H5ETM7voJqY67VU0tOS9iU1dEd2sr2ZJ0SYtbB6TaaYTko3nw7MAI4lhcDeA+wlaTiApA0l7QD8ARgqac+8fB1JOxb291PgVuB6SWsD9wJjJG2Whw0PKZTdBPi//PmosnpdRhoC/HlEvFGnczUzswaqV0M1A9gcmBURTwOvAjPyUN0E4FpJ80kN14iI+BswHviOpHmkRu0fijuMiO+ThhGvBp4m9ZRmAb/Ly0smkhq0GUD5PajJwCC6GfaTtJukZaQG8GJJi2o5eTMz6zttHaEkqQM4LyL26bZwjRyhZGZWu55EKLXcA7/1IulU0lt7fW/KzKwfa7kHfvPr6BfWUP4KSePz58skvQ8gIs6JiG0i4i5JEyT9SNLp+VX1xT+n99W5mJlZ77VVjyoiunw1R57ufnZXZay93TF1u7rvc799/1j3fZrZai3Xo8rWknSppEWSbpO0gaRRku6RNF/STZLeVr6RpGn5vlSfJlmYmVnjtGpDtT3w44jYEXge+BRwFSnlfGdS4sXXOtu4r5IsnExhZtZ4rdpQLY6IUpz1HNLDw4Mj4s687Ergg11s//ckizwV/rrCui2BKZIWACcBpee3Lgc+kz9XTLKIiEsioiMiOoYOHdqT8zIzsxq1akO1svD5DWBwD/bRVZLFjyJiJPAvwPqQkiyAYpLFb3pwTDMzq7P+MpniBeA5SftExAzg08CdXZS/Fzhf0mbAi6QHeeflddUkWVztJIv25IkPZv1Pf2moIDUqF0naEHiclC1YUUQ8JWkiKcniKVKSxVp59URSksX/kZIyim/7nUwa8ut1gK2ZmdVHWydT1KqWJAsnU5iZ1c7JFL3gJAszs9bUqpMpGq6YZNHsupiZ2WpuqMzMrKW13NCfpGnAiRHR6Q0gSROAjog4rk7HXI/0QPFo4Fng0IhYUo99W/NNnDixX+/fbKBzjyo5BnguIoYD5wHfaXJ9zMws63VDJelkScfnz+dJmpo/7yfpGkn7S5ol6QFJ10salNePlnSnpDmSpuTYo+J+3yLpSknfzN/7MrvvIFLaBcANwH6SVOFcHaFkZtZg9ehRTQdK07k7gEH5dfF7kzL5zgDGRcT7gdnACXn9BcD4iBhNii8qppqvDUwCHomIM/oqu6/gXcBSgIh4nfSA8WblhRyhZGbWePW4RzUHGC1pY1L00QOkBmsf0gO07wNm5g7KuqSHcN8D7ATcnpevRXowt+Ri4Of5tRxQyO4DkHQdsENetyVwXW7M1gUW5+WXA78CfkAn2X0Fb+o90XkEk5mZNVCvG6qIeE3SElJSxN3AfGAsKUh2MXB7RBxe3EbSSGBRROzZyW7vBsZK+l5EvFo6VCdlLwC+HxGTJY0hJU8QEUslFbP7uno+ahmwFbBM0tqkmKU/d1He+hFPdjDr3+o1mWI6cGL+7wzgWGAuKaJoL0nDASRtKGkH4A/AUEl75uXrSNqxsL+fAreSoo7WJmX3jZG0WR42PKRQtprsvp93k903ubDteGBqOLLDzKwl1KuhmgFsDsyKiKeBV4EZeahuAnCtpPmkhmtEfvXGeOA7kuaRGrV/KO4wIr5PGka8Gnia1FOaBfwuLy+ZSGrQZgDl96AmA4PoPrvvp8Bmkh4DTgBOrfbEzcysb7V11l8t2X21ctafmVntnPVX4Ow+M7P20LYNVUScA5xTXCbpdNa8vwVwfWF2ofVzy06d0fBjbnlO3TvsZlbQr5IpJA2W9KXC9zGSfl3DLl4g3bPahfRs16iIOFvSiPxQ8kpJJ9a73mZm1nP9qqEivZL+S92W6txM0sO/T5Qt/zNwPPDdXuzbzMz6QMMbKknDJD0s6TJJCyVNkjRO0swce7S7pImSLpc0TdLjpYgm0lDedpLmSjo3Lxsk6Ya8z0mVoo9KIuLBSmGzEfFMRNwPvNZN3R2hZGbWYM3qUQ0Hzgd2BkYAR5Ail04ETstlRgAfBnYHvpafnzoV+GMesjspl9sV+AopAWNbCjmA9eYIJTOzxmtWQ7U4IhbkTL5FwB35AdsFwLBc5paIWJnz+Z4B3tHJvu6LiGV5X3ML25uZWRto1qy/lYXPqwrfV7G6TsUyb9B5XastZwOAZ+CZtZ/+NpniJWDjZlfCzMwap181VBHxLCmJfWFhMkXVJB0vaRkpcX3TOUt1AAAOPElEQVS+pMvy8nfm5ScAZ0haJumtda28mZn1SFtHKPUlRyiZmdWuJxFK/apHZWZmA09bTjyQdBPw7rLFp0TElGbUxxrje4ce0JTjfvW6WsJRzKxWbdNQSRoMHBERF0bEJ/NLFE+MiKp+e+XXhJQmarydNO394L6prZmZVaudhv56Fa8UEfvkB4lHkd57dWPdamZmZj3WUg1VM+OVCnXYGNgX+GWFdY5QMjNrsJZqqLJmxyt9kpSU8WL5CkcomZk1Xis2VM2OVzocuLY3J2BmZvXTipMpmhavJGkzUi/tk9VW1lqHZ9+ZtadW7FH1VD3ilQ4Bfh0Rr9ahPmZmVgdt01D1Nl4pOwwP+5mZtRRHKPWQI5TMzGrnCCUzM2s7rTiZok85Xqk9/fjYqU079pcv2rdpxzYbCPpVj0rSYElfKnwfI6nqqV6SjgNGArsA4wpJFEMkzc9/7pa0S/1rb2ZmPdGvGip6GZMEzATGAU+ULV8MfCgidgbOAi7pxTHMzKyOGt5QNTMmKSIejIglFZbfHRHP5a/3kF6sWKnujlAyM2uwZvWomh2T1JVjgN9UWuEIJTOzxmtWQ9XsmKSKJI0lNVSn9HQfZmZWX82a9de0mKTOSNoZuAz4aH542PoRz7wza1/9bTJFPWKS3kTS1qT3T306Ih6p9/7NzKzn+lVD1duYJEnHS1pGmiwxX9JledWZwGbAhXmihiMnzMxahCOUesgRSmZmtXOEkpmZtZ22iVCSNIz0io6dHJPUNx4a8d5mV6Elvffhh5pdBbO21jYNVVFE+MWHZmZtot2G/taSdKmkRZJuk7RBTrfoAJA0RNKS/HmCpF9KulnSYknHSTpB0oOS7pG0aVPPxMzMgPZrqLYHfhwROwLPA5/qpvxOpFSM3YGzgb9GxK7ALOAz5YUdoWRm1njt1lAtjoi5+fMcuk+p+J+IeCkilgMvADfn5cWEjL9zhJKZWeO12z2q8pSKDYDXWd0gr99F+c4SMizzpAEza4Z261FVsgQYnT+Pb2I9zMysBwZCQ/Vd4IuS7gaGNLsyZmZWGydT9JCTKczMaudkCjMzazsNa6gk3SppcA3lh0la2Jd16uLYLzfjuGZm9mYNm9kWER9r1LGsb4y8cmSzq9CSFhy1oNlVMGtrdetRSTpZ0vH583mSpubP+0m6RtKSnAwxTNJD5QkSuexoSfMkzQK+XNj3jpLuy6/gmC9p+7yfhyVdmZfdIGnDwn7ulDRH0hRJm+fl20n6bV4+Q9KIvPzdkmZJul/SWfW6JmZm1nv1HPqbDuyTP3cAgyStA+wNzCgr21mCxM+A4yNiz7LyxwLnR8SovO9lefl7gEsiYmfgReBL+ZgXAOMjYjRwOSl1AuAS4F/z8hOBC/Py84GfRMRuwJ96egHMzKz+6tlQzQFGS9qY9ODsLFKjsg9vbqjelCAhaRNgcETcmZdfXSg/CzhN0inANhHxSl6+NCJm5s/XkBrF95CikW6XNBc4A9hS0iDgH4Dr8/KLgc3ztnsB11Y47hocoWRm1nh1u0cVEa/lwNejgbuB+cBYYDugPNKgUoKEgIpz5SPivyTdC3wcmCLpc8DjFcpH3s+i8l6ZpLcCz+deWcXDdHmCqR6XkHpldHR0eF6/mVkD1HsyxXTSkNpnSXl53wfmRERI6nLDiHhe0guS9o6Iu4AjS+skbQs8HhE/zJ93JjVUW0vaMyJmAYcDdwF/AIaWluehwB0iYlFOST8kIq5XqtDOETEPmAkcRuqVHYlV5EkDZtYM9Z6ePoM0nDYrIp4GXuXNw35dORr4cZ5M8Uph+aHAwjxkNwK4Ki9/CDhK0nxgU9J9pr+RopK+I2keMJc05AepETomL18EHJSX/z/gy5LuBzap5YTNzKxv9dtkiuIbfZtxfCdTmJnVzskUZmbWdvrtqywiYglpdp+ZmbUx96jMzKyl9dseVb1JOpv0+vm3RcSgZten5Uz0HJNOTXyh2TUwa2vuUa12M7B7sythZmZramR6+kaSbslZfgslHdpFJt/nc+7ePEm/KGT4HZK3nSdpel62vqSfSVog6UFJY/PyCZJuzNl+j0r6z67qFxH3RMRT3ZyDkynMzBqskT2qjwBPRsQueUr5b+k8k+/GiNgtInYhPSt1TF5+JvDhvPzAvOzLABExkvTQ75WS1s/rRpGewRoJHCppq96cQERcEhEdEdExdOjQ3uzKzMyq1Mh7VAuA70r6DvBr4DlWZ/IBrAWUejQ7SfomMBgYBEzJy2cCV0j6OXBjXrY3qcEjIh6W9ASwQ153R0S8ACDp98A2wNI+O0MzM6u7Rr6P6hFJo4GPAd8GbqdCJl92BXBwRMyTNAEYk/dxrKQ9SJl/cyWNImX7daY8U9CTR3rKEwbMrEkaeY9qC+CvEXEN8F1gD3ImX16/jqQdc/GNgadyTl8x82+7iLg3Is4EVgBbkfIFj8zrdwC2JuX9mZlZG2hkD2MkcK6kVcBrwBeB14Ef5ld8rA38gJTB9x/AvcATpCHDjfM+zpW0PakXdQcwD3gYuEjSgry/CRGxsrsQ3HJ5ssURwIaSlgGXRcTEnp+umZnVQ7/N+ms2Z/2ZmdXOWX9mZtZ2BtzkgvwCxvXKFn86IvyyJTOzFtTwhirP4rstIp7M35cAHRGxos7HuZV0zwngiIi4ECAi9qhQdhtJc0hT5NcBLoiIi+pZn3LDTr2lL3dvDbTknI83uwpmba0ZQ38TgC3qsSNJnTa0EfGxiHie9CzWl7rZ1VPAP+TX1O8BnJpnKZqZWZN121BJOlnS8fnzeZKm5s/7SbpG0v6SZkl6QNL1kgbl9WfmGKSFki5RMh7oACZJmitpg3yYf83bL5A0Im+/kaTL8z4elHRQXj4hH+dm4DZJm0uanve3UNI+udwSSUOAc4Dt8vpzK51jRPwtIkrPXK3X2XVxhJKZWeNV06OaDuyTP3cAg/LzTXuTpo6fAYyLiPcDs4ETctkf5RiknYANgAMi4oZc5siIGBURpdfNr8jb/wQ4MS87HZgaEbsBY0lT0zfK6/YEjoqIfUnDe1Nyb2gX0qvni04F/piPd1JnJylpq/xK+6XAd0pDk0WOUDIza7xqGqo5wGhJG5OSHmaRGqx9gFeA9wEzJc0FjiLFFAGMlXRvfr5pX2DHN+15tVIc0hxgWP68P2kIbi4wDVif9DAvwO0R8ef8+X7gaEkTgZER8VIV5/QmEbE0InYGhgNHSXpHT/ZjZmb11e1kioh4LU94OBq4G5hP6uFsBywmNRqHF7fJobAXkiZJLM2NyPp0rjTsVow5EvCpiFgjZSJHKP2lUL/pkj5IilW6WtK5EXFVd+fVmYh4UtIiUkN8Q0/30x3fgDczq061kymmk4bkpgMzgGNJQ2z3AHtJGg4gacMcY1RqlFbke1bjC/t6idVJE12ZQrp3pbzvXSsVkrQN8ExEXAr8FHh/WZFujydpy9L9MklvA/bCMUxmZi2h2oZqBrA5MCsingZeBWZExHLSLL5r8/2de4ARebbdpaR7WL8kDc+VXEGKPCpOpqjkLNJU8fmSFubvlYwhBdQ+CHwKOL+4MiKeJQ1NLuxsMgXwXuBeSfOAO4Hv+rkqM7PW4AilHpK0nJRFOFANIQUDm69FOV+P1Xwt1jQE2CgiapqN5obKekTS7FrzutqVr8WafD1W87VYU0+vx4CKUJI0Eri6bPHKSmkVZmbWGgZUQ5XvO41qdj3MzKx6Tk+3nrqk2RVoIb4Wa/L1WM3XYk09uh6+R2VmZi3NPSozM2tpbqjMzKyluaGyqkjaVNLtkh7N/31bJ+XeyA9zz5U0udH17EuSPiLpD5Iek3RqhfXrSbour79X0rDG17JxqrgeEyQtL/w8fK4Z9exr+S0Pz+RggkrrJemH+TrNl1SentNWqrgeYyS9UPi5OLO7fbqhsmqdCtwREdsDd+TvlbySk+pHRcSBjate35K0FvBj4KOkIObDJb2vrNgxwHMRMRw4D/hOY2vZOFVeD4DrCj8PlzW0ko1zBfCRLtZ/FNg+//kC6S0R7ewKur4ekJKNSj8X3+huh26orFoHAVfmz1cCBzexLs2wO/BYRDweEX8D/pt0TYqK1+gGYL9SVmUbquZ6DAgRMR34cxdFDgKuiuQeYLCkzRtTu8ar4nrUzA2VVesdEfEUQP7v2zspt35+ueQ9ktqpMXsX6V1lJcvysoplIuJ14AVgs4bUrvGquR4An8rDXTdI2qoxVWs51V6rgWRPSfMk/UZSV6+AAgbYA7/WNUm/A95ZYdXpNexm6/yqlG2BqZIWRMQf61PDpqrUMyp/tqOaMu2imnO9Gbg2IlZKOpbU29y3z2vWegbSz0U1HgC2iYiXJX2MFFy+fVcbuKGyv4uIcZ2tk/S0pM0j4qk8bPFMJ/t4Mv/3cUnTgF2BdmiolgHFHsGWQPlboEtllklaG9iEOg+BtJBur0d+c0HJpbTxPbtuVPOzM2BExIuFz7dKulDSkIjoNLzXQ39WrcmkNziT//ur8gKS3iZpvfx5COm9Xr9vWA371v3A9pLeLWld4DDSNSkqXqPxwNRo3yfqu70eZfdhDgQeamD9Wslk4DN59t8HgBdKw+gDkaR3Ft4zuDupHXq2q23co7JqnQP8XNIxwP8ChwBI6gCOjYjPkd7rdbGkVaQfvnMioi0aqoh4XdJxpBd6rgVcHhGLJH0DmB0Rk0kv7rxa0mOkntRhzatx36ryehwv6UDgddL1mNC0CvchSdeS3os3RNIy4Gukd+kRERcBtwIfAx4D/kp6W3rbquJ6jAe+KOl14BXgsO7+QecIJTMza2ke+jMzs5bmhsrMzFqaGyozM2tpbqjMzKyluaEyM7OW5obKzMxamhsqMzNraf8fIbX3nyXvaokAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt8VHed//HXJzcugRIgAYFwSVug0gu3lN7UVtsqVgVbL23V3Var/NStrtff1p9afdTV3VV3veKluv2p64W2brtixVJqqba1WMKllGsbQiEhQAIJgZD7zGf/mEk6TSdkgJycmcn7+XjMI3POfOfM52TgvHPO95zvMXdHREQEICfsAkREJH0oFEREpIdCQUREeigURESkh0JBRER6KBRERKSHQkFERHooFEREpIdCQUREeuSFXcCpKi4u9hkzZoRdhohIRtmwYcNhdy/pr13GhcKMGTOoqKgIuwwRkYxiZntTaafDRyIi0kOhICIiPQILBTO7x8zqzGxrH6+bmX3XzCrNbIuZLQiqFhERSU2Qewo/Axaf5PU3AzPjj2XADwOsRUREUhBYKLj7X4CGkzRZCvzCY9YBRWY2Kah6RESkf2H2KUwBqhOma+LzREQkJGGGgiWZl/Q2cGa2zMwqzKyivr4+4LJERIauMEOhBpiaMF0K1CZr6O53u3u5u5eXlPR77YWISFaJRp1/fmg7zx86HvhnhRkKK4G/j5+FdCnQ5O4HQqxHRCQtPbztID99cg/P1TQF/lmBXdFsZr8BrgKKzawG+BKQD+DuPwJWAdcBlUAL8P6gahERyVSRqPPvj+zi3AmjePv84LtdAwsFd7+5n9cd+IegPl9EJBs8uGk/u+tP8KP3LSA3J1lX7MDSFc0iImmqvSvCt9Y8z4VTxvCm8181KJ+pUBARSVP3rq9m/9FWPvum2ZgFv5cACgURkbTU2hHhe49VcknZOF47s3jQPlehICKShtbuqqP+eDsfe8PMQdtLAIWCiEhaenjrQcYVFnDZOeMH9XMVCiIiaaa9K8LanXVc++qJg3LGUSKFgohImvnr7iMcb+/iTRdMHPTPViiIiKSZR7YdZNSwPC4/Z/A6mLspFERE0kgk6qzZfoirZpcwPD930D9foSAikkY27G3kcHMHiy8YnIvVelMoiIikkdXbDlKQl8NVsyeE8vkKBRGRNOHuPLz1IK89t5hRwwIbmu6kFAoiImlix4Hj7D/aOmjjHCWjUBARSRPP7DkCwGsGcViL3hQKIiJpYsO+o0waM5zJRSNCqyHQUDCzxWa2y8wqzeyOJK9PN7M/mdkWM3vczEqDrEdEJJ1t3NvIguljQ60hsFAws1xgOfBmYA5ws5nN6dXsm8Av3P0i4C7gX4KqR0QknR1samP/0VYWTsvSUAAWAZXuXuXuHcAKYGmvNnOAP8Wfr03yuojIkLBxXyNA9u4pAFOA6oTpmvi8RM8C74g/vx4YbWaDOySgiEga2LC3kWF5OcyZdFaodQQZCsmG9vNe058BrjSzTcCVwH6g6xULMltmZhVmVlFfXz/wlYqIhGzjvkbmlhZRkBfu+T9BfnoNMDVhuhSoTWzg7rXufoO7zwc+H5/X1HtB7n63u5e7e3lJSUmAJYuIDL62zghb9zeFfugIgg2F9cBMMyszswLgJmBlYgMzKzaz7ho+B9wTYD0iImlp6/4mOiPOwmwOBXfvAm4HVgM7gPvcfZuZ3WVmS+LNrgJ2mdnzwETgq0HVIyKSrjbsjXUyz59WFHIlEOjgGu6+CljVa96dCc9/C/w2yBpERNLdxn2NzBg/kuJRw8IuRVc0i4iEyd3ZsPdoWvQngEJBRCRU1Q2tHG5uT4v+BFAoiIiEasO+BgAWhHwlczeFgohIiDbtO0phQS6zJo4OuxRAoSAiEqpN+44yd2oRuTnJrvcdfAoFEZGQtHZE2HHgWNocOgKFgohIaJ7b30RX1NPi+oRuCgURkZB0j4w6X3sKIiKyKX7R2rjCgrBL6aFQEBEJgbuzcd/RtNpLAIWCiEgo9h9tpf54OwvSqD8BFAoiIqHYuO8okF79CaBQEBEJxaZ9jQzPz+G8V6XHRWvdFAoiIiHYuO8oF5UWkZebXpvh9KpGRGQIaOuMsL22Ka0uWuumUBARGWTbao/RGUmvi9a6BRoKZrbYzHaZWaWZ3ZHk9WlmttbMNpnZFjO7Lsh6RETSwaZ96XOntd4CCwUzywWWA28G5gA3m9mcXs2+QOw2nfOJ3cP5B0HVIyKSLjbtO0rp2BFMGD087FJeIcg9hUVApbtXuXsHsAJY2quNA2fFn48BagOsR0QkLWza15h2p6J2CzIUpgDVCdM18XmJvgy8z8xqiN3L+WPJFmRmy8yswswq6uvrg6hVRGRQHGxqo7apjflT0+/QEQQbCskGB/de0zcDP3P3UuA64L/M7BU1ufvd7l7u7uUlJSUBlCoiMjjSuT8Bgg2FGmBqwnQprzw8dBtwH4C7Pw0MB4oDrElEJFSbqo9SkJfD+ZPHhF1KUkGGwnpgppmVmVkBsY7klb3a7AOuBjCzVxMLBR0fEpGstWlfIxdMPouCvPS8IiCwqty9C7gdWA3sIHaW0TYzu8vMlsSbfRr4kJk9C/wGuNXdex9iEhHJCh1dUbbUNKVtJzNAXpALd/dVxDqQE+fdmfB8O3BFkDWIiKSLnQeP0d4VTdv+BNAVzSIig2ZTfGTUdBzeoptCQURkkGza18jEs4YxaUz6XbTWTaEgIjJINu47yvypYzFLdsZ+elAoiIgMgsPN7exraEnr/gRQKIiIDIrN3f0J09O3PwEUCiIig2L93gbyc40L0vSitW4KBRGRQbCuqoG5pUWMKMgNu5STUiiIiATseFsnW/c3cdk548MupV8KBRGRgK1/sYFI1LnsbIWCiMiQt66qgYLcnLTvZAaFgohI4J7efYT504oYnp/e/QmgUBARCVRTayfbapu4NAMOHYFCQUQkUM/saSDqZEQnMygUREQCta7qCMPyctL+SuZuCgURkQA9vfsIC6ePZVhe+vcnQMChYGaLzWyXmVWa2R1JXv+WmW2OP543s6NB1iMiMpiOtnSw4+CxjOlPgABvsmNmucBy4Fpi92teb2Yr4zfWAcDdP5nQ/mPA/KDqEREZbOuqGvAM6k+AYPcUFgGV7l7l7h3ACmDpSdrfTOyWnCIiWeGvuw8zIj+XuaWZ0Z8AwYbCFKA6YbomPu8VzGw6UAY8FmA9IiKDxt1Zu6uOK84dT0Fe5nTfBllpsrtIeB9tbwJ+6+6RpAsyW2ZmFWZWUV9fP2AFiogEZXf9CaobWrlq9oSwSzklQYZCDTA1YboUqO2j7U2c5NCRu9/t7uXuXl5SUjKAJYqIBGPtzjoAXn+eQqHbemCmmZWZWQGxDf/K3o3MbDYwFng6wFpERAbV2l11zJ44milFI8Iu5ZQEFgru3gXcDqwGdgD3ufs2M7vLzJYkNL0ZWOHufR1aEhHJKMfbOln/YkPG7SVAgKekArj7KmBVr3l39pr+cpA1iIgMtqcqD9MZcV4/O/MOd2dOl7iISIZYu7Oe0cPzWJgBQ2X3plAQERlA3aeivm5WCXm5mbeJzbyKRUTS2LbaY9Qdb+cNGXYqajeFgojIAFq7sw4zuDID+xNAoSAiMqAe3naQeVOLKB41LOxSTotCQURkgLx4+ATbao/xlgsnhV3KaVMoiIgMkD88dwCA6xQKIiLyhy0HWDCtiMkZdhVzIoWCiMgA2HP4BNsPHMvovQRQKIiIDIhVWXDoCBQKIiID4qEtB1g4fWxGHzoChYKIyBmrqm9mRxYcOgKFgojIGXvp0NGrQq7kzCkURETOgLvzwKb9XDxjLJPGZPahI1AoiIickY37GqmqP8G7Fk7tv3EGSDkUzOw1Zvb++PMSMysLriwRkcxw7/pqRhbkct1Fmd+fACmGgpl9Cfgn4HPxWfnAL1N432Iz22VmlWZ2Rx9t3m1m281sm5n9OtXCRUTC1tzexUNbDvDWiyYxalig9ywbNKmuxfXAfGAjgLvXmtnok73BzHKB5cC1QA2w3sxWuvv2hDYziQXNFe7eaGaZOdasiAxJq7YcoKUjwo0XZ8ehI0j98FFH/B7KDmBmhSm8ZxFQ6e5V7t4BrACW9mrzIWC5uzcCuHtdivWIiITu3opqzi4pZMG0zLvDWl9SDYX7zOzHQJGZfQh4FPhJP++ZAlQnTNfE5yWaBcwys6fMbJ2ZLU62IDNbZmYVZlZRX1+fYskiIsGprGtmw95GbiyfipmFXc6ASenwkbt/08yuBY4Bs4E73X1NP29L9lvyJJ8/E7gKKAWeMLML3P1or8+/G7gboLy8vPcyREQG3f0V1eTmGDcsKA27lAGVUijEDxc95u5rzGw2MNvM8t298yRvqwESD7SVArVJ2qyLL2ePme0iFhLrU14DEZFB1tYZ4b6Kaq4+bwIlozPzZjp9SfXw0V+AYWY2hdiho/cDP+vnPeuBmWZWZmYFwE3Ayl5t/gd4PYCZFRM7nFSVYk0iIqF4YON+Gls6+cBrsu/M/FRDwdy9BbgB+J67Xw/MOdkb3L0LuB1YDewA7nP3bWZ2l5ktiTdbDRwxs+3AWuCz7n7kdFZERGQwRKPOPU/t4YIpZ3FJ2biwyxlwqZ6SamZ2GfBe4LZU3+vuq4BVvebdmfDcgU/FHyIiae/PL9RTWdfMt26cm1UdzN1S3VP4R+AO4IH4X/tlwGPBlSUikp7ueXIPE0YP4y0XTg67lECkuqfQAkSBm83sfcTOLNJZQCIypOw6eJwnXjjMZ980m4K87Bw6LtVQ+BXwGWArsXAQERly/vPJKobn5/DeS6aFXUpgUg2Fenf/faCViIiksZrGFh7ctJ+bLp5G0ciCsMsJTKqh8CUz+ynwJ6C9e6a7PxBIVSIiaeb7j1ViGB99/TlhlxKoVEPh/cB5xEZH7T585IBCQUSy3t4jJ7h/Qw1/d+n0rLiRzsmkGgpz3f3CQCsREUlT3/1TJXk5xkevyu69BEj9lNR1ZnbSi9VERLJRVX0zD26K7SVMOGt42OUELtU9hdcAt5jZHmJ9Ckbs2rOLAqtMRCQNfOdPLzAsL5cPD4G9BEg9FJIOaS0iks227m9i5bO1/J/XnUPxqOwa+K4vqQ6dvTfoQkRE0om787VVOygakc9HhsheAqTepyAiMqSs3VXHX3cf4RPXzGLMiPywyxk0CgURkV66IlG+tmonZcWFvCeLr15ORqEgItLLivXVVNY1c8ebzyM/d2htJofW2oqI9KO5vYtvP/o8i8rG8cY5E8MuZ9AFGgpmttjMdplZpZndkeT1W82s3sw2xx8fDLIeEZH+3P3n3Rxu7uD/XffqrLxfQn9SPSX1lJlZLrAcuJbYvZjXm9lKd9/eq+m97n57UHWIiKTq0LE2fvLEHt560STmTS0Ku5xQBLmnsAiodPcqd+8AVgBLA/w8EZEz8u1Hn6crGuWzb5oddimhCTIUpgDVCdM18Xm9vcPMtpjZb81saoD1iIj06YVDx7l3fTXvu3Q608cXhl1OaIIMhWQH43rfre33wIz4cBmPAj9PuiCzZWZWYWYV9fX1A1ymiAj828M7KSzI42NvmBl2KaEKMhRqgMS//EuB2sQG7n7E3bvvz/ATYGGyBbn73e5e7u7lJSUlgRQrIkPX07uP8OiOOj7y+nMYV5i9N9BJRZChsB6YaWZlZlYA3ASsTGxgZpMSJpcAOwKsR0TkFaJR56urtjOlaAQfuKIs7HJCF9jZR+7eZWa3A6uBXOAed99mZncBFe6+Evi4mS0BuoAG4Nag6hERSebBTfvZuv8Y375xHsPzc8MuJ3Tm3vswf3orLy/3ioqKsMsQkSzQ2hHh9d98nIlnDePBj15BTk72XpdgZhvcvby/drqiWUSGrJ8+UcXBY2184a1zsjoQToVCQUSGpEPH2vjhn3ez+PxXcfGMcWGXkzYUCiIyJH3loe10RZ3PXXde2KWkFYWCiAw5T7xQz0NbDvAPV507pC9US0ahICJDSltnhC/+z1bKigv58FVnh11O2gnslFQRkXT04z9X8eKRFn552yUMy9MpqL1pT0FEhoyq+maWP17J2+ZO5jUzi8MuJy0pFERkSOiMRPnkvZsZWZDLF9/y6rDLSVs6fCQiQ8LytZU8W9PED967gAlnDQ+7nLSlPQURyXqbq4/yvccquX7+FK67cFL/bxjCFAoiktVaOyJ86t7NTBw9jC8vOT/sctKeDh+JSFb7yh+2s+fICX512yWMGZEfdjlpT3sKIpK1Ht56kF//bR/LXnc2l5+rs41SoVAQkax0oKmVOx7YwkWlY/j0tUP3nsunSqEgIlknEnU+de+zdHRF+c5N8ynI06YuVepTEJGs8/O/vsjTVUf4xjsvoqxYYxudikDj08wWm9kuM6s0sztO0u6dZuZm1u8NIERETuZgUxv//sgurppdwjsXloZdTsYJLBTMLBdYDrwZmAPcbGZzkrQbDXwc+FtQtYjI0HHXQ9voijp3LbkAM90451QFuaewCKh09yp37wBWAEuTtPsK8HWgLcBaRGQIWLuzjlXPHeTjV89k2viRYZeTkYIMhSlAdcJ0TXxeDzObD0x194dOtiAzW2ZmFWZWUV9fP/CVikjGa+2IcOfKrZw7YRQfeq2GxD5dQYZCsv0273nRLAf4FvDp/hbk7ne7e7m7l5eUlAxgiSKSLb716PNUN7TylaUX6GyjMxDkb64GmJowXQrUJkyPBi4AHjezF4FLgZXqbBaRU/XMngZ+8kQV77lkGpedMz7scjJakKGwHphpZmVmVgDcBKzsftHdm9y92N1nuPsMYB2wxN0rAqxJRLJMc3sXn75/M1PHjuTz12lI7DMVWCi4exdwO7Aa2AHc5+7bzOwuM1sS1OeKyNDy1T9sp6axlf9491wKh+nSqzMV6G/Q3VcBq3rNu7OPtlcFWYuIZJ/Hdh7iN89U8+Erz6F8xriwy8kK6o0RkYx06Fgbn7l/C+e9ajSfvHZm2OVkDYWCiGScSNT5xIrNtHZE+P57FjAsLzfskrKGDsCJSMZZvrayZ2yjcyeMCrucrKI9BRHJKH+rOsK3H32et8+brLGNAqBQEJGMcbi5nY+v2MS0cSP55+sv1NhGAVAoiEhGiEadT967mcaWTpa/dwGjdPppIBQKIpIRlq+t5IkXDvPlt53P+ZPHhF1O1lIoiEja++vuw3zr0edZOm8yNy+a2v8b5LQpFEQkre2ub+ajv9pIWXEhX1M/QuAUCiKStuqPt3Pr/3+GXDPuufViDWMxCPQbFpG01NLRxW0/X0/98XZWLLuM6eN1r+XBoD0FEUk7bZ0RPvLLjWzd38T3b17AvKlFYZc0ZGhPQUTSSltnhA/9ooInKw/zrzdcyDVzJoZd0pCiUBCRtNHS0cUHf17B01VH+Po7LuJd5TrTaLApFEQkLTSc6GDZLyrYuK+R/3j3XK6fryEswhBon4KZLTazXWZWaWZ3JHn9w2b2nJltNrMnzWxOkPWISHqqrGvm+h88xZb9TXzv5gUKhBAFFgpmlgssB94MzAFuTrLR/7W7X+ju84CvA/8RVD0ikp6eqjzMDT94ihPtXaxYdilvuWhS2CUNaUEePloEVLp7FYCZrQCWAtu7G7j7sYT2hYAHWI+IpJGuSJTv/ukFvr+2kpkTRvPTW8qZOm5k2GUNeUGGwhSgOmG6BrikdyMz+wfgU0AB8IYA6xGRNFHd0MIn7t3Mhr2NvHNhKV9ecr4GuEsTQX4Lya5Ff8WegLsvB5ab2XuALwC3vGJBZsuAZQDTpk0b4DJFZLBEos4v1+3lG6t3YcB3b57PkrmTwy5LEgQZCjVA4vlkpUDtSdqvAH6Y7AV3vxu4G6C8vFyHmEQy0PbaY3zuwed4tvoor51ZzNeuv1CHi9JQkKGwHphpZmXAfuAm4D2JDcxspru/EJ98C/ACIpJVDja18a01z3P/hmrGFRbwnZvmsWTuZA1sl6YCCwV37zKz24HVQC5wj7tvM7O7gAp3XwncbmbXAJ1AI0kOHYlIZmo40cF/PlnFfz65h0jUufXyMj5+9bkUjSwIuzQ5iUB7dtx9FbCq17w7E57/Y5CfLyKDr/ZoKz95oooVz1TT2hlh6bzJfOaNs3WoKEOou19Ezpi783TVEX61bh+rtx3EgaXzJvORK89h5sTRYZcnp0ChICKnrfZoK7/bXMtvN1Szu/4EY0bkc8vlM7j18hnaM8hQCgUROSVHWzr449aD/G7zfv62pwF3WDh9LN9817m89aJJDM/PDbtEOQMKBRHpV0tHF2u2H2Ll5lr+8kI9nRGnrLiQT1w9i+vnT2HaeO0VZAuFgogkFYk6T1Ye5sGNNTyy/RAtHREmjRnO+68oY8ncyZw/+SydVpqFFAoi8jKVdc08uKmGBzbu50BTG2NG5LN03hTePm8yF88YR06OgiCbKRREhKr6Zv649SC/f7aWnQePk2Nw5awSvvjWOVz96gkMy1M/wVChUBAZgtq7Imzce5THd9WxZschqupPALEO4y+9bQ5vuXASE84aHnKVEgaFgsgQcaS5nZXP1vLYzjrWv9hAW2eU/Fzj0rPHc8tlM7hmzkSmFI0Iu0wJmUJBJIt1RqI8trOO+ytqeHxXHV1RZ+aEUdx08TSuOLeYS88ex+jh+WGXKWlEoSCShaobWrivopp711dTd7ydCaOHcdtrynjHwlJm6QpjOQmFgkiWONbWyR+fO8ADG2MXleUYXDV7Au9ZNI2rZpeQlxvoLdklSygURDLYwaY2Ht1xiDXbD/H07iN0RKKcXVzIp6+dxQ0LS9VHIKdMoSCSQRpPdLD+xQb+uvsIT1Ue5oW6ZgBmjB/JLZdP5y0XTWZu6RhdVCanTaEgksYaT3TEAmD3YdbvaegJgRH5uVxcNo53LizlDedN4NwJoxQEMiAUCiJppKmlk2debOCZPUdYV9XA1tom3GHUsDzKZ4zl7fOnsKhsHHNLiyjIUx+BDLxAQ8HMFgPfIXbntZ+6+7/2ev1TwAeBLqAe+IC77w2yJpF00h0CT+8+wrqqI+w4eAx3KMjLYd7UIj55zSyuOLeYuaVj1FEsgyKwUDCzXGA5cC1QA6w3s5Xuvj2h2Sag3N1bzOwjwNeBG4OqSSRMkaizu76ZLTVNbNrXSMWLjew6dByIhcDCaWP5xNWzuPTsccydWqQhqCUUQe4pLAIq3b0KwMxWAEuBnlBw97UJ7dcB7wuwHpFB0RWJUtPYyp4jJ9hd18wLh5p5vu44uw4ep6UjAsQOBy2YPpa3zZ1E+YxxzFMISJoIMhSmANUJ0zXAJSdpfxvwx2QvmNkyYBnAtGnTBqo+kdN2vK2T6oZW9jW0UNPYwt4jLextaGHfkRPUNLbSFfWetuMLC5g1cTTvLp/KRaVjuKh0DGXFo8jVaKOShoIMhWT/4j3JPMzsfUA5cGWy1939buBugPLy8qTLEBkoHV1R6pvbqTvWRt3xdg4cbaW2qY39ja1UN7ZQ3dBCY0vny94zenge08eP5PzJY7juwknMKC6krLiQs4sLGT9qWEhrInLqggyFGmBqwnQpUNu7kZldA3weuNLd2wOsR4aYts4IR1s6OdrawbHWLo61dtLU2snxtk6OtcWmG1s6OdrSQUNLBw0nOmho7uB4e9crljUsL4fJRSMoHTuCCy6cxNSxI5k2LvaYOm4EY0bk65RQyQpBhsJ6YKaZlQH7gZuA9yQ2MLP5wI+Bxe5eF2AtNLXG/vP3xRJ2bFL5v52sjZm9bPeou0333MT3vOztL5tvPe1y4ssziy/Dup/HX7PYz9jjpXmZtHGKRp3OaJTOiNPZFaUjEqWjK0p7V4S2ziitnRHaOiO0dERo6eiK/WyPcLy9ixPtXTS3dXG8vZPjbV0cb+viWFvseVNrJx1d0ZN+9oj8XMaOzGdsYQFjRxZQOnYk4wsLGFdYQMnoYUwYPYwJo4czqWg44wsLMur3KnK6AgsFd+8ys9uB1cROSb3H3beZ2V1AhbuvBL4BjALuj/+H2+fuS4KoZ8Uz+/iXP+4MYtFp56Ww6BUaOb0DJPa8u30sgLqD5aWA6pkPxLMp/jkvbSTdY0f1HHCHqDuRqBONOhF3ItHYvK5IlEjU6YzGnkfP4GDgyIJcRg3LY/TwPEYNz+es4XlMKRrBWSPyGD08n6KR+RSNKGDMiPyex+jheZwV/5mvUzxFXiHQ6xTcfRWwqte8OxOeXxPk5yd6/XkTKBmd/NiuJ2yY+tpGeUKjpG0cPOGV7ubeazo275Xtetom2bi6d097z3scJ5rweiTqPe175nlso+zEXo/Et8DdG+zuZUbjG+zuz3R/6bWeeQnTvX8JjvfsyQDkJgZSjsWmc4y8HCM3/sjLNfJzcmI/c3MoyI09L8jLYVheLgV5OQzPy2FEQS7D83MZkZ9L4bA8CgtyGVGQS2FBnm4LKRKAIXNF86yJozVksIhIP7T/LCIiPRQKIiLSQ6EgIiI9FAoiItJDoSAiIj0UCiIi0kOhICIiPRQKIiLSwxKv1M0EZlYPDOTd2YqBwwO4vDBpXdJTtqxLtqwHDM11me7uJf01yrhQGGhmVuHu5WHXMRC0LukpW9YlW9YDtC4no8NHIiLSQ6EgIiI9FArxO7plCa1LesqWdcmW9QCtS5+GfJ+CiIi8RHsKIiLSY8iFgpl9xcy2mNlmM3vEzCb30S4Sb7PZzFYOdp2pOIV1ucXMXog/bhnsOlNhZt8ws53x9XnQzIr6aPeimT0XX+eKwa4zFaewLovNbJeZVZrZHYNdZ3/M7F1mts3MombW59ktGfKdpLouaf2dAJjZODNbE///vMbMxvbR7vS2Ye4+pB7AWQnPPw78qI92zWHXOhDrAowDquI/x8afjw279iR1vhHIiz//N+Df+mj3IlAcdr1nui7EblG7GzgbKACeBeaEXXuvGl8NzAYeB8pP0i4TvpN+1yUTvpN4nV8H7og/v+Mk/1dOaxs25PYU3P1YwmQhfd+BM+2luC5vAta4e4O7NwJrgMWDUd+pcPdH3L0rPrkOKA2znjOR4rosAirdvcrdO4AVwNLBqjEV7r7D3XeFXcdASHFd0v47iVsK/Dz+/OfA2wdy4UMuFAAQ5LP+AAAFd0lEQVTM7KtmVg28F7izj2bDzazCzNaZ2YD+0gdSCusyBahOmK6Jz0tnHwD+2MdrDjxiZhvMbNkg1nS6+lqXTPxe+pJp30lfMuU7mejuBwDiPyf00e60tmFZeY9mM3sUeFWSlz7v7r9z988DnzezzwG3A19K0naau9ea2dnAY2b2nLvvDrDspAZgXZLd3T6UvaP+1iXe5vNAF/CrPhZzRfx7mQCsMbOd7v6XYCru2wCsS1p8L6msRwoy5jvpbxFJ5qXd/5VTWMxpbcOyMhTc/ZoUm/4a+ANJQsHda+M/q8zscWA+seONg2oA1qUGuCphupTYcdVB19+6xDvB3wpc7fGDokmW0f291JnZg8R2+Qd9AzQA61IDTE2YLgVqB67C1JzCv6+TLSMjvpMUpMV3AidfFzM7ZGaT3P2AmU0C6vpYxmltw4bc4SMzm5kwuQTYmaTNWDMbFn9eDFwBbB+cClOXyroAq4E3xtdpLLFO0NWDUd+pMLPFwD8BS9y9pY82hWY2uvs5sXXZOnhVpiaVdQHWAzPNrMzMCoCbgLQ8y+1kMuU7SVGmfCcrge6zCG8BXrEXdEbbsLB70gf7Afw3sX+0W4DfA1Pi88uBn8afXw48R+zsg+eA28Ku+3TXJT79AaAy/nh/2HX3sS6VxI7nbo4/fhSfPxlYFX9+dvw7eRbYRuywQOi1n866xKevA54n9tdb2q0LcD2xv57bgUPA6gz+Tvpdl0z4TuI1jgf+BLwQ/zkuPn9AtmG6ollERHoMucNHIiLSN4WCiIj0UCiIiEgPhYKIiPRQKIiISA+FgmQdM2s+g/feHh8h0+Pnd3fPNzP7bvy1LWa2IOG1SWb2UD/LnWFmJz1/P5U2fdT7/lN5j8jJKBREXu4p4Bpgb6/5bwZmxh/LgB8mvPYp4CeDUt0r3UNshFyRAaFQkKwV/+v+G2a2NT7e/43x+Tlm9oP4+PoPmdkqM3sngLtvcvcXkyxuKfALj1kHFMWHGAB4B/BwfNkzzOwJM9sYf1yepK5bzex3ZvZwfOz+xKFJcs3sJ/HaHjGzEfH3fMjM1pvZs2b232Y2Ml5vC/CimS0amN+aDHUKBclmNwDzgLnE/vr/RnxDfgMwA7gQ+CBwWQrLSjqCppmVAY3u3h6fXwdc6+4LgBuB7/axvEXERradB7zLXrrxy0xgubufDxwlFjgAD7j7xe4+F9gB3JawrArgtSmsg0i/snJAPJG41wC/cfcIcMjM/gxcHJ9/v7tHgYNmtjaFZfU1guYkoD5hXj7wfTObB0SAWX0sb427HwEwswfiNf0PsMfdN8fbbCAWXgAXmNk/A0XAKF4+flUdcF4K6yDSL4WCZLNkG/KTzT+ZvkbQfBUwPGH+J4mNrTOX2J54Wx/L6z2+TPd0e8K8CDAi/vxnwNvd/Vkzu5WXj3w7HGhNYR1E+qXDR5LN/gLcaGa5ZlYCvA54BngSeEe8b2EiL9/A9mUl8PfxfopLgSaP3eDkeV76ax5gDHAgvhfyd8Ru8ZjMtRa71+4IYnfOeqqfzx8NHDCzfGKHnRLNInNHJpU0o1CQbPYgsRFknwUeA/6vux8kNrpsDbEN6Y+BvwFNAGb2cTOrIbYnsMXMfhpf1ipi97euJHam0UcB3P0EsNvMzo23+wFwi5mtI7axPtFHbU8C/0VsFNX/dvf+bnj/xXida3jlEOlXAI/2836RlGiUVBmSzGyUuzeb2Xhiew9XxAPjdJZ1PbDQ3b+QYvtbid08/vbT+bxey5oPfMrd/+5MlyUC6lOQoeshMysCCoCvnG4gALj7g/FwCUMxsb0IkQGhPQUREemhPgUREemhUBARkR4KBRER6aFQEBGRHgoFERHpoVAQEZEe/wtJdO2Tl8dwjgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cv of rmse: 0.51707091585331\n",
      "rmse_train: 0.38909527953583645\n",
      "rmse_test: 0.4266888489288939\n",
      "r2_score_train: 0.8486048634429293\n",
      "r2_score_test: 0.8282552085292978\n"
     ]
    }
   ],
   "source": [
    "\n",
    "lasso = LassoCV()\n",
    "\n",
    "lasso.fit(X1_train,y1_train)\n",
    "\n",
    "#交叉验证得到最佳超参数\n",
    "alpha = ridge.alpha_\n",
    "print 'alpha:',alpha\n",
    "\n",
    "\n",
    "#绘制重要系数\n",
    "coefs=pd.Series(lasso.coef_,index=X1_train.columns)#Wj\n",
    "\n",
    "#正系数值最大的10个特征和负系数最小值(绝对值大)的10个特征\n",
    "imp_coefs=pd.concat([coefs.sort_values().head(10),coefs.sort_values().tail(10)])\n",
    "imp_coefs.plot(kind=\"barh\")\n",
    "plt.title(\"Coefficients in the Lasso Model\")\n",
    "plt.show()\n",
    "\n",
    "\n",
    "mses = np.mean(lasso.mse_path_,axis=1)\n",
    "plt.plot(np.log10(lasso.alphas_),mses)\n",
    "plt.xlabel('log10(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()\n",
    "#交叉验证估计的测试误差\n",
    "mse_cv = np.mean(lasso.mse_path_,axis=0)\n",
    "rmse_cv = np.sqrt(mse_cv)\n",
    "print 'cv of rmse:',min(rmse_cv)\n",
    "\n",
    "\n",
    "#预测\n",
    "y_train_pred_lasso = lasso.predict(X1_train)\n",
    "y_test_pred_lasso = lasso.predict(X1_test)\n",
    "y_test_pred_lasso += mean_diff\n",
    "\n",
    "#测试误差\n",
    "rmse_train = np.sqrt(mean_squared_error(y1_train,y_train_pred_lasso))\n",
    "rmse_test = np.sqrt(mean_squared_error(y1_test,y_test_pred_ridge))\n",
    "\n",
    "print 'rmse_train:',rmse_train\n",
    "print 'rmse_test:',rmse_test\n",
    "\n",
    "\n",
    "r2_score_train = r2_score(y1_train,y_train_pred_lasso)\n",
    "r2_score_test = r2_score(y1_test,y_test_pred_lasso)\n",
    "\n",
    "print 'r2_score_train:',r2_score_train\n",
    "print 'r2_score_test:',r2_score_test\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评价：训练集要好点，测试集稍差"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef_lasso</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.299985</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>0.399123</td>\n",
       "      <td>mnth_9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.107857</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>0.172972</td>\n",
       "      <td>mnth_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.084883</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>0.181575</td>\n",
       "      <td>mnth_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>0.133200</td>\n",
       "      <td>mnth_8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.099216</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>0.085897</td>\n",
       "      <td>mnth_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.115274</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>0.111285</td>\n",
       "      <td>mnth_10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>0.006151</td>\n",
       "      <td>mnth_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.069855</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>-0.162792</td>\n",
       "      <td>mnth_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>-0.222486</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>-0.121306</td>\n",
       "      <td>mnth_7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.113631</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>-0.210659</td>\n",
       "      <td>mnth_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>-0.215730</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>-0.285308</td>\n",
       "      <td>mnth_11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>-0.231573</td>\n",
       "      <td>1.836371e+12</td>\n",
       "      <td>-0.310139</td>\n",
       "      <td>mnth_12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.278460</td>\n",
       "      <td>1.274409e+12</td>\n",
       "      <td>0.475001</td>\n",
       "      <td>weathersit_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>1.274409e+12</td>\n",
       "      <td>0.200737</td>\n",
       "      <td>weathersit_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>-0.859227</td>\n",
       "      <td>1.274409e+12</td>\n",
       "      <td>-0.675738</td>\n",
       "      <td>weathersit_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0.001532</td>\n",
       "      <td>4.314330e+11</td>\n",
       "      <td>0.117634</td>\n",
       "      <td>weekday_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>-0.184355</td>\n",
       "      <td>4.314330e+11</td>\n",
       "      <td>-0.099789</td>\n",
       "      <td>weekday_0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>0.027278</td>\n",
       "      <td>2.237385e+11</td>\n",
       "      <td>0.109745</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>-0.156466</td>\n",
       "      <td>2.237385e+11</td>\n",
       "      <td>-0.127590</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0.008139</td>\n",
       "      <td>2.076944e+11</td>\n",
       "      <td>0.042899</td>\n",
       "      <td>weekday_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.076944e+11</td>\n",
       "      <td>0.031574</td>\n",
       "      <td>weekday_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.076944e+11</td>\n",
       "      <td>0.031428</td>\n",
       "      <td>weekday_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>-0.025182</td>\n",
       "      <td>2.076944e+11</td>\n",
       "      <td>-0.017253</td>\n",
       "      <td>weekday_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>-0.117621</td>\n",
       "      <td>2.076944e+11</td>\n",
       "      <td>-0.106493</td>\n",
       "      <td>weekday_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1.398925</td>\n",
       "      <td>1.316086e+00</td>\n",
       "      <td>0.928808</td>\n",
       "      <td>temp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>1.046493</td>\n",
       "      <td>1.047598e+00</td>\n",
       "      <td>1.047439</td>\n",
       "      <td>yr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.444552</td>\n",
       "      <td>5.619551e-01</td>\n",
       "      <td>0.801991</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.300667</td>\n",
       "      <td>4.348705e-01</td>\n",
       "      <td>0.407402</td>\n",
       "      <td>season_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.001802</td>\n",
       "      <td>4.165967e-02</td>\n",
       "      <td>0.051654</td>\n",
       "      <td>season_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.039142</td>\n",
       "      <td>-7.505030e-02</td>\n",
       "      <td>-0.042814</td>\n",
       "      <td>season_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.530573</td>\n",
       "      <td>-4.038885e-01</td>\n",
       "      <td>-0.416243</td>\n",
       "      <td>season_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>-0.524837</td>\n",
       "      <td>-6.221201e-01</td>\n",
       "      <td>-0.561283</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>-0.520763</td>\n",
       "      <td>-7.012057e-01</td>\n",
       "      <td>-0.586317</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    coef_lasso       coef_lr  coef_ridge       columns\n",
       "12    0.299985  1.836371e+12    0.399123        mnth_9\n",
       "8     0.107857  1.836371e+12    0.172972        mnth_5\n",
       "9     0.084883  1.836371e+12    0.181575        mnth_6\n",
       "11    0.000000  1.836371e+12    0.133200        mnth_8\n",
       "6     0.099216  1.836371e+12    0.085897        mnth_3\n",
       "13    0.115274  1.836371e+12    0.111285       mnth_10\n",
       "7    -0.000000  1.836371e+12    0.006151        mnth_4\n",
       "5    -0.069855  1.836371e+12   -0.162792        mnth_2\n",
       "10   -0.222486  1.836371e+12   -0.121306        mnth_7\n",
       "4    -0.113631  1.836371e+12   -0.210659        mnth_1\n",
       "14   -0.215730  1.836371e+12   -0.285308       mnth_11\n",
       "15   -0.231573  1.836371e+12   -0.310139       mnth_12\n",
       "16    0.278460  1.274409e+12    0.475001  weathersit_1\n",
       "17   -0.000000  1.274409e+12    0.200737  weathersit_2\n",
       "18   -0.859227  1.274409e+12   -0.675738  weathersit_3\n",
       "25    0.001532  4.314330e+11    0.117634     weekday_6\n",
       "19   -0.184355  4.314330e+11   -0.099789     weekday_0\n",
       "31    0.027278  2.237385e+11    0.109745    workingday\n",
       "30   -0.156466  2.237385e+11   -0.127590       holiday\n",
       "24    0.008139  2.076944e+11    0.042899     weekday_5\n",
       "22    0.000000  2.076944e+11    0.031574     weekday_3\n",
       "23    0.000000  2.076944e+11    0.031428     weekday_4\n",
       "21   -0.025182  2.076944e+11   -0.017253     weekday_2\n",
       "20   -0.117621  2.076944e+11   -0.106493     weekday_1\n",
       "26    1.398925  1.316086e+00    0.928808          temp\n",
       "32    1.046493  1.047598e+00    1.047439            yr\n",
       "27    0.444552  5.619551e-01    0.801991         atemp\n",
       "3     0.300667  4.348705e-01    0.407402      season_4\n",
       "1     0.001802  4.165967e-02    0.051654      season_2\n",
       "2    -0.039142 -7.505030e-02   -0.042814      season_3\n",
       "0    -0.530573 -4.038885e-01   -0.416243      season_1\n",
       "29   -0.524837 -6.221201e-01   -0.561283     windspeed\n",
       "28   -0.520763 -7.012057e-01   -0.586317           hum"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(columns), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T)), \"coef_lasso\":list((lasso.coef_.T))})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
